towards improved process efficiency in vaccine innovation ... 13 - towards improved... · towar ove...

Towards improved process efficiency in vaccine innovation: the Vaccine Innovation Cycle as a validated, conceptual stage-gate model

van de Burgwal LH, Ribeiro CdS, van der Waal MB, Claassen E

Vaccine 2018; https://doi.org/10. 1016/ j. vaccine. 2018. 10. 061

Chapter 13

13

240

Abstract

Continuing investments in vaccine innovation are insufficiently translated into market entries of novel vaccines. This innovation paradox is in part caused by stakeholders lacking complete understanding of the complex array of steps necessary for vaccine development and collaboration difficulties between the wide variety of stakeholders involved. Models providing cross-domain understanding can improve collaboration but currently lack both comprehensibility and granularity to enable a prioritized view of activities and criteria important for vaccine innovation. Key opinion leaders (KOLs) were asked to contribute to the definition of a vaccine innovation cycle (VIC). In a first step, 18 KOLs were interviewed on the stages (activities and results) and gates (evaluation criteria and outcomes) of vaccine innovation. This first description of the VIC was subsequently validated and refined through a survey among 46 additional KOLs. The VIC identifies 29 distinct stages and 28 corresponding gates, distributed in ten different but integrated workstreams, and comprehensibly depicted in a circular innovation model. Some stage-gates occur at defined moments, whereas the occurrence and timing of other stage-gates is contingent on a variety of contextual factors. Yet other stage-gates continuously monitor internal and external developments. A gap-overlap analysis of stage-gate criteria demonstrated that 5 out of 11 criteria employed by vaccine developers correspond with criteria employed by competent (regulatory) authorities. The VIC provides a comprehensive overview of stage-gates throughout the value chain of vaccine innovation. Its cyclical nature highlights the importance of synchronizing with unmet needs and market changes, and conceptualizes the difference between incremental and radical vaccine innovation. Knowledge on the gap between internal and external criteria will enhance the viability of newcomers to the field. The VIC can be used by stakeholders to improve understanding and communication in forming collaborative alliances and consortia. Such a boundary-spanning function may contribute to the reduction of process inefficiencies, especially in public-private partnerships.

Towards improved process efficiency in vaccine innovation

13

241

13.1 Introduction

Vaccines are considered one of the greatest achievements in medicine, since they are a cost-effective strategy to decrease the global burden of infectious diseases (122, 557-561). So far, 26 infectious diseases have been targeted by vaccines, and 24 other diseases have been the focus of vaccine development (561). Despite these continuing efforts, product innovation is far from efficient and increased investments in research and development (R&D) are not followed by an increase in new market entries of novel vaccines (43, 562). This ‘vaccine innovation paradox’ (563) is in part inherent to the complex and challenging nature of vaccine development. Vaccine innovation requires the inclusion of a wide variety of expertise, including societal demand articulation (564), basic and applied research (565), intellectual property management (566, 567), market and manufacturing (568), quality assurance and regulatory compliance (569, 570). The commercialization process of new medical entities is like “trying to thread a long string through 10.000 sequential needles”, while missing any checkpoint could mean the development of the entity is doomed (571).

With substantial investments needed to obtain this wide variety of expertise (37) and the significant uncertainty accompanying vaccine development (572), only a handful of large companies have managed to accumulate the resources needed for bringing vaccines into market deployment (573). Small- and medium-sized companies (SMEs) rarely progress to later phases of clinical development (574). Moreover, they struggle with a limited viability, resulting in a lack of affordable, available and accessible vaccines that address societal unmet needs (575, 576). This is irrespective of the importance of SMEs for radical innovations such as vaccines that build upon scientific advances or use novel marketing approaches to deliver new product-market combinations. Indeed, SMEs bridge R&D gaps between basic discovery and early clinical development and large companies build upon this progress to meet regulatory demands (574). Still, SMEs usually develop vaccines for high-income markets due to the greater potential for return on investments, rather than focusing on products that are designed for use in low-income countries’ immunization programs. Moreover, in the context of radical innovation, collaboration with and between scientists from a wide variety of disciplines is essential to build upon recent advances (123). Furthermore, alignment of needs and strategies between vaccine developers, governments and NGOs is necessary to address changing societal and market unmet needs (573). The importance of collaboration extends into the development of agreed evidence to inform novel licensure pathways and overall policy recommendations and even beyond to design post-marketing surveillance international collaboration, engagement and communication between regulators and those using or recommending vaccines is needed (577). To address the innovation paradox and develop radical innovations into novel vaccines, improved collaboration between stakeholders from different disciplines and across different sectors is therefore essential (219, 573, 574, 578, 579).

Accordingly, efforts have been made towards co-development of vaccines through public-private partnerships (574, 580). Such collaborations, however, are not easy to establish and a range of challenges are present in these stakeholder collaborations (566). One of these challenges is related

Chapter 13

13

242

to the specialization of expertise in silos, leading stakeholders to have in-depth insight into their own set of activities while failing to understand the complexity of the total value chain (581). As a consequence, stakeholders from different domains might lack reciprocal understanding and appreciation of each other’s significance in optimizing the innovation process (547, 582). To better comprehend decision-making, insight into the accompanying criteria and societal needs that are used to evaluate these steps is necessary(583, 584). However, whereas some have identified that the vaccine industry evaluates the business case and the scientific feasibility to set priorities for R&D (573), a thorough description of criteria used to evaluate the progress of vaccine innovation activities is lacking. Finally, stakeholders from different domains have diverging norms and priorities, complicating cross-domain collaborations (30, 150, 521). Public and private stakeholders from academia, public health institutes, regulators, industry, SMEs, health care professionals, vaccine users, and policymakers, therefore have to bridge their own perspective in order to effectively collaborate with each other (101, 570, 574, 585).

DE

CBAAgenda Setting

CommercialPublicMediaNGO / WHOGovernment

CBA

Gene functionIncrementalRadical

15 MARKET INTROGRESSION

16 PRODUCT ACCESIBILITY

3 UNMET (MEDICAL) NEED

2 SOCIETAL IMPACT

1 (RE)EMERGING INFECTION

4 DEMAND ARTICULATION

BA

Puri�cationProduction

13 UPSCALING

FEDCBA

Ecological ImpactRegulatoryQualityClinical (E�cacy) Clinical (Safety) Pre-clinical

HG

(Access Framework)AccessabilitySocietal Impact

12 EVALUATION

BA

Prototype Pilot

11 PROOF-OF-CONCEPT

BA

Funding Recommendation

14 MARKET ACCESS STEPS

Development Discourse

Market & Society Discourse

VaccinesTherapeuticsDiagnosticsKoch's Postulates

7 PROOF OF PRINCIPLE

6 PATHOGEN DISCOVERY

5 SAMPLING

8 INVENTION

Scienti�c Discourse9 IP DECISION FRAMEWORK

QPON

Service for FeeConsultancyTrainingContract Research

WVU

T

SR

Spin-outLicenceSale

Guidelines

IP PoolConsortium Building

FEDC

BA

Social MediaLayman’s PublicationCoursesReport

Conf. ProceedingsOral Presentations / Scienti�c Publication

10 REALIZATION

MLKJI

HG

ProductSoftwareDesignsIPDatabaseInstruments / Tools

Expert Role

A Accessibility

EDC

AdoptionAvailabilityA�ordability

EDC

Cold ChainShelf lifePackaging

IP ProtectionNecessary?

NO

YESFEBA

TrademarkTrade secretDefensive publicationPatent

G Company brandingDCBA

FIGURE 13.1 | Valorisation and Technology Transfer Cycle, adapted from Ribeiro et al., 2018 (587).


13

243

Models including cross-domain representations provide an overview of the whole vaccine innovation process and can thereby facilitate dialogue among stakeholders and contribute to a collective search for innovative solutions (574, 584). Such models have been shown to improve stakeholder collaboration and efficiency in many sectors (586). Existing models within the vaccine industry, however, do not provide the necessary level of detail for all stakeholders to understand their respective contribution in the innovation processes (563). The well-known pharmaceutical value chain (PVC) provides a linear description of R&D stages for pharmaceutical compounds from target to launch but doesn’t provide insight into the accompanying processes such as manufacturing and market feedback. The recently developed “Drug Discovery, Development and Deployment Maps (4DM)” provides a more thorough representation of therapeutic development processes for biologicals (584) but leaves out non-technical processes and is not specific for vaccines. Both models do not elucidate the role that stakeholders from market and societal domains play in vaccine innovation. In the context of the Nagoya Protocol to the Convention on Biological Diversity, these domains have become even more important, as recently shown for the sharing of isolates and data from microorganisms (587).

In contrast to the PVC and the 4DM, the Valorisation and Technology Transfer Cycle (VTTC) (265, 565, 587) is a circular model that includes market and society domains feeding into science and business development domains (see Figure 13.1). This model provides a more comprehensive understanding of the full range of domains involved in vaccine innovation and it cyclical nature allows for quality improvement based on existing expertise. Although relevant and usable for new players in the field, it does not yet provide the granularity needed to have a prioritized view of activities within the cycle. In addition, this model does not provide context on the criteria used to evaluate whether each stage of activities has been successfully completed. Insight into the processes, activities and criteria employed in each domain can foster mutual understanding and enhance recognition of each other’s work (582). More importantly, considering the interdependencies the different domains, such insight can help actors to anticipate needs, constraints and requirements in early development stages, influencing their ability to decide on resource allocation and continuity of innovation projects in an integrated product development approach (586). Therefore, in order to improve resource allocation and innovation effectiveness, here we provide granularity to the VTTC in terms of activities and criteria as prioritized by Key Opinion Leaders (KOLs) in vaccine innovation.

13.2 Methodology

Building upon the stage-gate model as a conceptual framework, the methodology for this study consisted of two phases. In the first phase, 18 qualitative interviews were held with key opinion leaders (KOLs) in the field to identify stages and gates within the vaccine development process. After data analysis and the compilation of a first draft of an integrated model, survey responses from 46 KOLs were collected and their feedback was incorporated into a final version of the Vaccine Innovation Cycle (VIC).

Chapter 13

13

244

13.2.1 Conceptual framework

Based on studies of R&D processes within major corporations, the Stage-Gate (SG) system by Robert Cooper cs. has become a widely implemented managerial tool for guiding new product projects from initial idea to market launch and beyond (588, 589). SG is based on the premise that often economically risky and technically uncertain R&D projects benefit from managing that risk through structured evaluation moments; so-called gates. Consisting of a set of predefined evaluation criteria, these gates enable fact-based decision-making that is likely to improve project effectiveness and efficiency (588).

The stage-gate model thus distinguishes stages, as periods in time during which information gathering activities are turned into deliverables; and gates, as the evaluation and decision-making moments following each stage. The comparison of the stages’ deliverables against gates’ criteria results in an outcome to progress, recycle, kill or hold the project, thereby decreasing the project’s uncertainties and risks (588). Here the combination of the stage-gate model and the VTTC was used to describe the activities and relevant criteria in the vaccine innovation process. The combination and integration of the stage-gate model with the VTTC has the potential to address the persistent need of a complete and contextualized conceptual model representing the vaccine innovation process, by describing its activities per stage-gate and therefore providing a complete overview of all steps and criteria in the vaccine innovation process.

13.2.2 Interview respondents

KOLs, characterised as influential individuals within vaccine innovation, were considered the most appropriate informants for constructing a conceptual process model for their specific field. KOLs were identified in three distinct ways. Firstly, a search for invited guest speakers at international symposia was conducted; secondly, a search for senior level professionals was conducted on the websites of large organizations active in vaccine innovation; and thirdly, a strategy of snowball sampling was adopted, in which participants were asked to identify additional KOLs from their network to participate in the current study. Respondents were identified through purposive sampling (590), enabling the selection of KOLs from all domains represented in the vaccine innovation process. Background analyses were conducted before reaching out to potential interviewees. Potential interviewees were considered eligible if their backgrounds clearly demonstrated expertise with the subject matter. KOLs wished to remain anonymous due to confidentiality concerns and therefore an anonymized list of KOLs is provided in Supplementary File S13.1. 18 out of 40 invited KOLs agreed to participate in the interviews (response rate = 45%).

13.2.3 Interviews

Participants were contacted via e-mail, informed about the nature of the study and invited to take part. Detailed information included the study aim, central concepts and interview structure. These introductionary topics were covered at the start of the interview as well, while participants were given the opportunity to state ambiguities and ask questions on the matter.


13

245

Interviews were conducted via telephone and started with an explicit, oral informed consent on data collection for the purpose of this study, with respondents being anonymized and transcripts not being made publicly available to prevent de-anomyization.

0

20

40

60

80

100

120

140

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Tota

l num

ber m

entio

ned

in in

terv

iew

s

Interviews

Gate-relatedcriteriaStage-relatedactivities

FIGURE 13.2 | Saturation of gate-related criteria and stage-related activities was reached after 15 interviews.

The semi-structured interviews were based on a topic list. This ensured a broadly similar structure among the various interviewees while allowing for follow-up questions to appear during the course of the interview to obtain information as in-depth as possible (167). The topic list included a number of proposed stages and gates per domain, emphasising and therefore starting with the domain to which the expertise of the concerned KOL related most. For the first interviews these were derived from a previous version of the VTTC (265), and for each subsequent interview findings from previous interviews were incorporated. Respondents were asked to comment on the activities taking place, the deliverables of each stage, the evaluation criteria, the results and outcomes and the actors involved. Interviews were conducted until saturation of stage-related activities and gate-related criteria was achieved, as evidenced by three subsequent interviews in which no new concepts were mentioned (see Figure 13.2).

Interviews were audio recorded and manually transcribed by the interviewer. In addition, they were summarized and sent to the interviewee for confirmation and optional comments or modifications. Finalised transcripts were examined thoroughly, while text fragments that appeared important to the study were underlined. Data was subsequently analysed by two independent researchers using the conventional content analysis approach (301). In this process, text fragments were coded and categories were deductively derived from the data. Finally, this led to the identification of 20 stages and 26 gates.

A short description of stage-related activities and deliverables were grouped next to gate-related criteria and outcomes in tables. The stages with their activities and the gates with their criteria were combined into a first graphical draft of the model, consisting of concentric lines. Stage-gates that occurred in a sequential manner were placed on one line, stage-gates that occurred in a parallel manner were grouped on different lines.

Chapter 13

13

246

13.2.4 Survey

The resulting description of the vaccine innovation process was validated through a survey among KOLs. E-mail addresses from 209 KOLs were collected. The sampling included the identification of experts who (1) participated as a guest speaker on international vaccine conferences in 2017; (2) were a member of the Strategic Advisory Group of Experts on Immunization (SAGE) for the WHO in the period 2012-2016 or (3) were mentioned as influencer of immunization programs across the globe on a blog by the Bill and Melinda Gates foundation in 2013.

The cross-sectional survey was developed in SurveyMonkey and started with an informed consent, a notice of confidentiality, and a time indication. After gathering respondent’s demographic data, the draft model was introduced and general responses regarding the understandability of the model were collected. To enable high-quality input, respondents were asked to indicate to which domain their expertise related the most. Subsequently, they were asked to choose a group of activities within that domain (e.g. “Scoping, exploration and pre-clinical” or “CMC” for the science domain; or “Preparation and actual manufacturing” or “Preclinical, clinical and market preparation” for the business development domain). For that group, respondents were asked to reflect on the semantic quality of the model (e.g. whether the information provided was correct, complete, relevant and sufficiently reflecting the course of actions in the real-world) and on the comprehensibility of the model (e.g. whether the model provided insights on the activities within their domain and met informational needs) (591). A pilot study with four respondents was conducted to ensure that all questions were clear and to verify how much time on average it would take to fill out the questionnaire.

After receiving an initial invitation, all KOLs received one general reminder and one final reminder to increase response rates (440). The first invitation was sent in September 2017 and the survey was closed in November 2017. At that moment, 64 responses had been collected, indicating a response rate of 31%, which is well within the norm for surveys (36 +/- 13%) (302). From all respondents who started the survey, 46 completed questions on one of the domains, indicating a completion rate of 72%. The demographic characteristics of the respondents of the survey are shown in Supplementary File S13.2. All data were treated confidentially and were processed anonymously.

Based on the comments on the semantic quality and comprehensibility of the model, the stage-related activities and gate-related criteria were reformulated, and some items were divided into separate stages or gates. Based on the comments regarding the comprehensibility of the model, changes to the layout were made and a revised model was drawn-up.

The finally identified criteria were analysed for their use by different stakeholders; either by stakeholders directly contributing to vaccine innovation (internal criteria), or by stakeholders evaluating whether progress meets predetermined guidelines set by competent authorities (external criteria). The criteria were analysed for their occurrence in different gates and a gap-overlap analysis of internal and external criteria was conducted.


13

247

13.3 Results

Building upon the proposed conceptual model, the interviewees indicated 20 stages and 26 gates that occurred throughout the value chain of vaccine innovation. In the survey, these were further refined into 29 distinct stages and 28 corresponding gates. The initial description of stage-gates was refined with comments gathered from the survey, and a final description of each stage and its corresponding gate, organized in ten different but integrated workstreams, is provided in section 13.3.1.

13.3.1 Description of stages and gates

Respondents from both interviews and survey indicated that some stages were part of a clear value chain and occurred in a fixed order. These stage-gates were classified as “defined”. The occurrence and timing of other stage-gates was contingent on a wide variety of contextual factors, therefore being classified as “undefined”. Yet other stages occurred continuously throughout the vaccine innovation process and were classified as “monitoring” stage-gates. In an integrated product development process, the different defined, undefined and monitoring stage-gates occurred in parallel workstreams and outcomes in one workstream influenced decision-making in other workstreams. In addition, some stages were executed in an iterative fashion.

Stages and gates at defined lociMore than half of the identified stage-gates occur at defined loci. These stage-gates occur in a relatively predictive order and timing. Some stages may run in parallel or iteratively, but the gates directly follow the stages and often successful completion of a certain gate is necessary before a subsequent stage can be started. A description of each of these stages and gates is provided in Table 13.1. Defined stages and gates comprise three groups of activities that consist of sequential steps but run in parallel: (1) research and development, (2) Good Manufacturing Practice (GMP) production and (3) market preparation, registration and introduction.

Chapter 13

13

248

TABLE 13.1 | The defined stages and gates as identified by the respondents belong to three groups of activities that run in parallel: (1) research and development, (2) cGMP production and (3) market preparation, registration and introduction.

Stage Gate

RESEARCH AND DEVELOPMENT

Stakeholder unmet needs assessment (S1) Individual stakeholders assess unmet needs to define opportunities for research and development. Unmet needs may be of a medical, technical or commercial nature.

Needs prioritization (D1) Based on strategic fit of unmet needs with the stakeholder in question, priorities for research and development are set. For-profit organizations additionally evaluate the commercial feasibility of solutions for unmet needs.

Scoping and preparation (S2)To address unmet needs, ideas are generated and developed, and research projects are prepared. Depending on the nature of the idea, partnerships may be developed.

Start exploration (D2)Based on an evaluation of strategic fit, technical feasibility and budget feasibility, resources are allocated to research projects. In addition, clinical trial feasibility, commercial feasibility and market feasibility are evaluated.

Exploration and discovery (S3)This stage consists of a range of research activities, including the elucidation of pathogenic mechanisms to identify targets and the generation of vaccine candidates. Later-stage developers may also be consulted to ensure compliance to later-stage criteria.

Lead identification (D3)Based on technical feasibility, manufacturing feasibility, commercial feasibility and budget feasibility, leads can continue to the next stage of preclinical development. Alternatively, projects may be terminated or recycled to test alternative vaccine candidates.

Early-stage preclinical (S4)Vaccine candidates are optimized and validated in simple animal models. In addition, later-stage developers, potential strategic partners and regulatory authorities may be consulted in this stage to shape development.

Candidate nomination decision (D4)Dependent on the technical feasibility, commercial feasibility and clinical trial feasibility (e.g. are correlates of protection for humans available), the project may progress to late stage preclinical development. Additional criteria in this gate are budget feasibility, operational feasibility and strategic fit.

Late-stage preclinical (S5)Vaccine candidates are tested in complex animalmodels to evaluate efficacy and/or immunogenicity, safety and toxicology and to find safe and efficacious doses, resulting in a preclinical information package

Initiate clinical trials (D5)Dependent on technical feasibility, commercial feasibility, clinical trial feasibility and budget feasibility, the project may progress into the initiation of a clinical trial.

Clinical trial application (S6)Based upon consultations with regulatory authorities, clinical trials are designed, resulting in an Investigational New Drug (IND), Clinical Trial Application (CTA) or similar application to the competent authorities. This consultation also informs the design of a regulatory strategy. Finally, the cost-benefit ratio of outsourcing to a Contract Research Organisation (CRO) is assessed.

First-in-man (D6)Approval to start clinical trials is contingent on the existence of a sufficiently relevant unmet need (ethical approval), adequate product performance, adequate clinical trial design and adequate quality. This is evaluated on the basis of preclinical and CMC (see S10) data in relation to draft leaflets / labels. In addition, the reliability and validity of the preclinical assays is evaluated. After approval, clinical trials can be started. Alternatively, authorities may conclude that certain steps need to be reiterated or that the project should be terminated.

RCT Phase I (S7)In a randomized controlled trial (RCT) Phase I study, the vaccine candidate is tested on human volunteers for safety, immunogenicity and vaccine pharmacokinetic parameters.

Pre-Phase II (D7)Based on technical feasibility, operational feasibility and budget feasibility (i.e. the costs for a powered clinical study), the project continues to Phase II, a refined Phase I is executed (e.g. with a higher dose), or the development is terminated.


13

249

Stage Gate

RCT Phase II (S8)The Phase II study can be a challenge study to study efficacy if sufficient treatment alternatives are available, or a vaccination study to study immunogenicity. After preparation with clinical sites, the candidate is tested for safety, efficacy and/or immunogenicity and mode of action parameters.

Pivotal development decision (D8)Based on technical feasibility, budget feasibility and registration feasibility, the project continues to Phase III, a refined Phase II is executed or the development is terminated.

RCT Phase III (S9)The Phase III study is prepared in alignment with the marketing department to support market adoption and in close collaboration with clinical sites to ensure recruitment of large numbers of participants. The vaccine candidate is tested on safety and efficacy and/or immunogenicity parameters. The resulting marketing and product dossiers are internally reviewed.

Registration decision (D9)Contingent on technical feasibility, commercial feasibility and registration feasibility, the candidate progresses into the market preparation and registration stage, additional Phase III studies are conducted or the development is terminated.

GMP PRODUCTION

CMC: Chemistry, Manufacturing and Control (S10)The necessary size of the production facility is assessed and manufacturing platforms fitting that size are developed, including up- and downstream processing. Assays for quality control are designed in consultation with regulatory authorities and the cost-benefit ratio of using contract manufacturing organizations (CMO) is assessed. Ultimately, the aim is to validate the manufacturing process.

CMC feasibility (D10)Dependent on the manufacturing feasibility, the operational feasibility of the dosage form, the commercial feasibility in terms of costs of goods and the budget feasibility in terms of the costs for the production plant, development of the candidate is continued into late-stage preclinical development, terminated or recycled until the above-mentioned conditions are sufficiently met.

Prepare manufacturing (S11)A manufacturing site is set up, either in-house or through a CMO, facilitating the upscaling of production under GMP conditions. A pilot production facility may be expanded to a larger scale and batch dossiers are delivered to evaluate batch-to-batch consistency.

Production feasibility (D11)Manufacturing feasibility, commercial feasibility and budget feasibility are evaluated before a manufacturing site is upscaled.

MARKET PREPARATION, REGISTRATION AND INTRODUCTION

Market preparation (S12)Marketing and pricing strategies are determined on the basis of information gathered in specific countries, the burden of disease and innovativeness of the product, health economics metadata, company ROI and pricing strategies and the willingness to pay. In addition, a market launch timeline is established and a business case for the new product is delivered.

Launch decision point (D12)Contingent on commercial feasibility, launch feasibility and the acceptability of liability risks, it is decided whether the product will be launched on the market. Outcomes are the launch specifics (e.g. countries, dates) and a timeline for registration.

Registration (S13)The dossier is submitted to the competent authority who assesses it on predefined criteria. In addition, a quality inspection of the end products is conducted.

Market authorization decision (D13)Contingent on adequate product performance, as evidenced in clinical trials with an adequate design, an adequate product quality supported by an adequate manufacturing process and supply system, full or conditional market approval may be given. Alternatively, the development may be reiterated after refinements to the product or the development may be terminated.

Chapter 13

13

250

Stage Gate

(National) implementation (S14)After market authorization (on a supranational level) is obtained, (national) implementation of the product is started. This includes the employment of the marketing strategy, and advocacy on the added value of the vaccine among stakeholders and governments. Competent authorities review dossiers submitted by the industry and consult stakeholders to assess reimbursement and/or inclusion in vaccination programs and reimbursement. In addition, negotiations on reimbursement and inclusion in vaccination programs are held.

(National) reimbursement (D14a)(National) reimbursement is contingent on adequate product performance for a sufficiently relevant unmet need. The pricing should be in line with the willingness to pay and with benchmarks of prices in other countries. These factors are evaluated in the context of the budget impact. In addition, the societal and political acceptability of reimbursement is evaluated. In turn, during the negotiations with competent authorities, the manufacturer evaluates the commercial feasibility in terms of ROI and their minimum price. This may result in full, partial or no reimbursement and consequently may influence the availability of the vaccine in certain territories.

Inclusion in vaccination program (D14b)Inclusion in vaccination programs is contingent on adequate product performance addressing an unmet need for a sufficiently large risk group and/or for society as a whole. In addition, the operational feasibility and the societal and political acceptability of vaccination programs is assessed. As a result, vaccines are included in vaccination programs with a procurement contract specifying the price, volume and duration of the contract, or not.

Market deployment (S15)Dependent on the outcomes of the implementation stage, vaccines are introduced into the market on a case by case basis or within the context of a vaccination program. In the first case, health care professionals are actively informed of the added value of the vaccine. In the latter case, vaccination programs are set up in a manner that ensure vaccination coverage.

This stage does not have its own gate, but is followed by post-marketing surveillance (M08).

Within these groups of defined stages and gates, successful completion of gates is necessary before a subsequent stage can be started. In addition, the start of some activities within one group is contingent upon successful completion of activities within another group. For instance, successful completion of the CMC gate is linked to preclinical development steps and the preparation of manufacturing should be completed before clinical trials are started. Successful completion of Phase III clinical trials is needed before market registration can be started. The general order and alignment of the stage-gates is presented in Figure 13.3. Nevertheless, iterations of activities within the same group occur, which may result in stages within one group running in parallel as well (e.g. after completion of RCT Phase II, an RCT Phase III and an RCT Phase II are started). Moreover, as indicated by some of the respondents the exact timing of parallel processes is contingent on contextual factors. In industry, for example, CMC is typically finished before early stage preclinical studies are started, whereas universities and SMEs only prioritize this gate during late-stage preclinical studies.


13

251

Linked gate

Monitoring stage-gates

Undefined stage-gates

Defined stage-gates

D3

D9

D4D2

D8

D5D1

D7

D6

D12

D10

D11

D14aD14b

D13

S01:Stakeholder

unmet needsassessment

S06:Clinical trialapplication

S07: RCTPhase I

S02:Scoping &

preparation

S03:Exploration

&discovery

S10: CMC:Chemistry, Manufacturing

and Control

S04:Earlystage

preclinical

S9:RCT

Phase III

S13:Registration

S14:National

implementation

S15:Market

deployment

S12:Market

preparation

S05:Late stagepreclinical

S11: Prepare

manufacturing

S8:RCT

Phase II

R&DGMP production

Market preparation, registration and introduction

FIGURE 13.3 | Defined stages and gates consist of three groups of activities occur in a relatively predictive order and timing. Different groups of activities partly run in parallel. The linking of the different groups of activities is indicated with purple arrows.

Stages and gates at undefined lociUndefined stages and gates occur adjacent to defined, core innovation stages and gates. Their occurrence and timing are contingent on a wide variety of factors, including the context in which the innovation project was initiated, changes in development and market landscapes and the specific characteristics of the vaccine candidate. The stages and gates are grouped in two separate types of activities. The first group relates to activities necessary for the business development of the innovation; and includes funding, IP protection, and strategic partnership. In contrast to the grouping of defined stage-gates, stages within this group do not necessarily occur sequentially, but they all contribute to the commercial viability of the organization active in research and development activities. The second group consists of one stage-gate related to manufacturing. A description of each of these stages and gates is provided in Table 13.2.

Chapter 13

13

252

TABLE 13.2 | The undefined stages and gates as identified by the respondents run in parallel to defined stages and gates. Their occurrence and timing are contingent on a wide variety of factors.

Stage Gate

FUNDING AND BUSINESS DEVELOPMENT

Funding (S16)This stage includes the acquisition of funding to support development steps, including non-dilutive funding, investors and early revenues generated through service. This stage is highly iterative in nature, since different sources of funding can be acquired at different steps in development. In addition, this stage involves the retaining of funding through updates to funders throughout the process and positive press releases to shareholders.

Funding (U16)Criteria to acquire funding differ per funding source and stage but include the technical feasibility of the project (thorough development plan, existing evidence and reasonable probability of success), a strategic fit with the funder, and the commercial, market and societal feasibility.

Scouting (S17)Identification of relevant findings in early stage R&D projects is followed by an assessment of the technical and market potential of such findings. This results in an invention disclosure form, a market analysis and a technical description of the finding.

Scouting (U17)Progression is contingent on the technical feasibility (added value and opportunities for further development) and commercial feasibility of the findings and includes progression to IP protection, spin-off company and/or partnering stages.

IP protection (S18)A thorough evaluation of the market, technical and patenting potential of potentially interesting results precedes drafting, filing and maintaining of patent applications. Results may be finalized after the filing of provisional applications. This stage occurs throughout development projects and, like S16, is highly iterative in nature.

IP protection (U18)The decision to secure IP protection is dependent on the technical and manufacturing feasibility of the candidate, the strategic fit with the organization from which the finding originated, the commercial feasibility and the budget feasibility of securing IP protection. Outcomes include patent applications and decisions for maintenance in specific territories. If criteria are insufficiently met, IP protection may (in part) depend on informal approaches like trade secrets.

Spin-off company (S19)When findings have a high commercial feasibility but further dedicated development outside of the organizational setting in which it was developed is necessary before being able to partner (S20), a spin-off company may be established.

Spin-off company (U19)Once the decision for starting a spin-off company is made, important criteria for establishing the spin-off company are commercial feasibility and strategic fit with an enthusiastic inventor who is willing to continue development and with a competent CEO.

Partnering (S20)Potential partners to improve development processes may be identified through a variety of channels. IP protection and confidentiality agreements often precede negotiations on respective rights (e.g. exclusivity, geography, right of first refusal, anti-shelving, publishing) and payments (upfront, milestones, royalties and secondary licensing).

Partnering (U20)Opportunities for partnering are evaluated on technical feasibility, commercial feasibility and strategic fit with the organization. In addition, they are contingent on agreement on rights and payments. A positive evaluation may result in a licensing agreement and/or a strategic partnership.

Acquisition (S21)Progress by biotech companies may spark interest by large vaccine companies who aim to acquire new innovations for their own development pipeline. In turn, small companies may seek acquisition to fund later-stage development. After successful negotiations, pivotal early development studies may be reiterated to ensure compliance to regulatory demands.

Acquisition (U21)Acquisition is contingent on technical and manufacturing feasibility, strategic fit, commercial feasibility, registration feasibility and budget feasibility. Pre-existing royalty deals or results that are easily reverse-engineered negatively impact acquisition decisions. Apart from successful acquisition, this stage may also result in reiteration of preclinical steps to improve outcomes. If acquisition efforts fail, biotech companies may decide to abandon the research project.


13

253

Stage Gate

MANUFACTURING

Manufacturing (S22)Manufacturing of clinical products includes effective up & downstream processing, quality assurance, quality control and the compilation of batch dossiers. Quality inspections ascertain consistent product manufacturing under GMP conditions.

Batch release (U22)Contingent on a GMP production facility, adequate quality assurance and quality control, as ascertained by a qualified person, batches of clinical product are approved and ready for clinical use. If either one of these criteria is not met, batches are discarded and production processes are evaluated.

Unlike defined stages and gates, the fulfilment of all undefined stage-gates is not required for each vaccine candidate. Moreover, the undefined stages and gates do not occur in a fixed order. Although the exact timing of these stage-gates cannot be defined beforehand and some stages (e.g. funding, IP protection or partnering) occur multiple times throughout a vaccine innovation project, a general order of these stage-gates and the alignment with the defined stage-gates is presented in Figure 13.4.

Linked gate



Defined stage-gates

U17

U18

U16

U19

U21

U20

U22

S16: Funding

S17: Scouting S18:

IP Protection

S19: Spin-off

company

S20: Partnering

S21:Acquisition

Funding and Business development

Manufacturing

S22:Manufacturing

FIGURE 13.4 | The undefined stages and gates as identified by the respondents run in parallel to the defined stages and gates. Their occurrence and timing are contingent on a wide variety of factors.

Manufacturing (S22) is an undefined gate because its timing is contingent on development and market conditions that result in changes in demand. Moreover, activities such as quality inspection occur at unannounced moments. Importantly, manufacturing only starts once the preparation of manufacturing stage (S11) is successfully completed. The undefined gates within the funding and business development group may require successful completion of one or multiple other stages but this is dependent on contextual factors.

Chapter 13

13

254

Monitoring stages and gatesMonitoring stage-gates take place in a continuous and iterative manner, rather than being preceded by the finalization of a previous stage-gate. Moreover, gates are less formal and at times represent outcomes of certain activities rather than fixed criteria against which deliverables of activities are evaluated. In general, five groups of activities can be identified within the monitoring stages and gates: (1) market monitoring, (2) innovation project monitoring, (3) portfolio monitoring, (4) public affairs monitoring, and (5) product monitoring. Here, each group refers to the focus of the activities. A description of each of these stages and gates is provided in Table 13.3.

A general alignment of the monitoring stages and gates with the defined and undefined stage-gates is presented in Figure 13.5.

D3

D9

D4D2

D8

D5D1

D7

D6

D12

D10

D11

D14aD14b

D13

S01:Stakeholderunmet needsassessment


S07: RCTPhase I

S02:Scoping &

preparation

S03:Exploration

&discovery


and Control

S04:Earlystage

preclinical

S9:RCT

Phase III

S13:Registration

S14:National

implementation

S15:Market

deployment

S12:Market

preparation


S11: Prepare

manufacturing

S8:RCT

Phase II

U17

U18

U16

U19

U21

U20

U22

S16: Funding

S17: Scouting S18:

IP Protection

S19: Spin-off

company

S20: Partnering

S21:Acquisition

S22:Manufacturing

R&D

GMP production


Mar

ket p

reparation, registration and introduction

Manufacturing

D3

D9

D4D2

D8

D5D1

D7

D6

D12D12

D10

D11D11

D14aD1414aD1D14bD144b

D13D13

S01:Stakeholderunmet needsassessment


S07: RCTPhase I

S02:Scoping &

preparation

S03:Exploration

&discovery


and Control

S04:Earlystage

preclinical

S9:RCT

Phase III

S13:Registration

S14:National

implementationmpmplementation

S15:Market

deploymentdedeployment

S12:Market

onionpreparation


S11: eePrepare

ngnmanufacturin

S8:RCT

Phase II

U17

U18

U16

U19

U21

U20

U22

S16: gggFunding

S17: Scouting S18:

IP Protection

S19: ffffSpin-off

cocompany

S2S20: PartneringnPartnerin

SSS21:onAcquisition

S22:ManufacturingMa r gnufactur g

R&R

D

GMP production


Mar

ket p

reparation, registration and introduction

Manuffaff cturing

M29

M23

M24M25

M26

M27M28

S23: Global

unmet needsassessment

S24: Demand

articulation

S25: Monitoring project

S26:monitoringpartnership

S28: Public Affairs

S27: Monitoring portfolio

S29: Postmarketingsurveillance

Linked gate



Defined stage-gates

Product monitoring

Market monitoring

Innovation project monitoring

Portfolio monitoring

Public affairs monitoring

FIGURE 13.5 | The monitoring stage-gates as identified by the respondents take place in a continuous and iterative manner. Post-marketing surveillance starts once the product is introduced to the market (S15).


13

255

TABLE 13.3 | The monitoring stages and gates as identified by the respondents take place in a continuous and iterative manner.

Stage Gate

MARKET MONITORING

Global unmet needs assessment (S23) On a global level, assessment of medical unmet needs for vaccines from a societal perspective, results in a list of vaccine-preventable unmet needs. This process is influenced by agenda-setting from societal stakeholders.

Global policy recommendations (M23) The unmet needs assessment leads to the formulation of (global) policy recommendations, including SAGE vaccines of Public Health Importance, target product profiles and suggestions for financing. Criteria include market, operational and societal feasibility, technical and manufacturing feasibility and commercial feasibility.

Demand articulation (S24)Demand articulation includes the prioritization of unmet needs and establishment of push & pull factors on the basis of advocacy by societal stakeholders and a review of the SAGE Vaccines of Public Health Importance.

Demand articulation (M24)Push & pull factors are put in place on the basis of criteria such as technical feasibility, societal feasibility and market feasibility.

INNOVATION PROJECT MONITORING

Monitoring project (S25) Monitoring of the progress of individual projects takes place on the basis of development plans with KPIs and deadlines. In addition, monitoring of market developments and consultation of regulatory stakeholders inform the evaluation of the progress of individual projects.

Monitoring project (M25)Depending on the progress of the project, its development is continued, terminated or improved. Such improvements may include updating of the development plan and/or updating of outcome criteria. Evaluation criteria include technical feasibility, budget feasibility, commercial feasibility, market feasibility, and continued strategic fit.

Monitoring partnership (S26)When a project is executed in collaboration with another partner, continued monitoring of the partnership is conducted. This includes managing expectations and partnership relations, monitoring compliance to agreements and assessing the continued added value of the partnership.

Monitoring partnership (M26)Depending on the added value of the partnership, in terms of productivity and reliability, the partnership is continued, improved or terminated.

PORTFOLIO MONITORING

Monitoring portfolio (S27)When multiple projects run in parallel, monitoring of the portfolio of projects is conducted to evaluate whether each project sufficiently progresses and contributes to the overall aim of the organization. This monitoring is informed by market developments.

Monitoring portfolio (M27)Depending on their relative performance and contribution, individual projects may be continued, terminated or improved. Evaluation criteria include technical feasibility, budget feasibility, commercial feasibility, market feasibility, and continued strategic fit. Internally started R&D projects may be preferred over externally started R&D projects.

PUBLIC AFFAIRS MONITORING

Public affairs (S28) Throughout development and market activities, monitoring of developments in the public domain takes place. Such developments include market developments which may influence accessibility, availability or affordability of new products, willingness to pay among patients and governments and developments relevant for vaccination programs.

This stage does not have its own gate, developments analysed in this stage inform other monitoring stages (such as monitoring projects, monitoring portfolios) as well as the design of defined development gates

Chapter 13

13

256

Stage Gate

PRODUCT MONITORING

Post-marketing surveillance (S29) Once vaccines are introduced into the market, post-marketing surveillance ensures timely recognition of adverse events following immunizations (AEFI), adequate vaccine quality, vaccination effectiveness and the gathering of market feedback on market and operational feasibility. In addition, such surveillance may lead to the identification and/or validation ofnovel correlates of protection or safety parameters.

Post-marketing surveillance (M29) Contingent on adequate product performance in terms of quality, safety and effectiveness, market registration is continued, suspended or withdrawn. In addition, market feedback may result in improved market deployment strategies.

13.3.2 Gate criteria

Next to identifying and grouping stage-gates involved in vaccine development, an analysis of the criteria employed in each stage-gate was conducted. In general, the criteria for the defined gates can be defined in two types.

External criteria, confined to fixed evaluation points by competent authorities, are used to assess whether vaccine candidates have adequately addressed the requirements for continuation into the next stage. Throughout these gates adequate product performance is a prominent checkpoint. Other prominent checkpoints include adequate product quality, adequate relevance / unmet need and adequate clinical trial design. Importantly, although some criteria are only applied at one gate, they can still abort successful innovation. A detailed overview of external criteria per gate is provided in Supplementary Material S13.3.

Internal criteria are used by stakeholders contributing to vaccine innovation to ensure that the vaccine candidate makes sufficient progress through internal gates before further resources are committed and/or they are submitted for external evaluation. The majority of internal criteria apply to all three types of gates, as shown in Table 13.4.

TABLE 13.4 | Applicability of criteria to different types of gates.

Criteria topic Defined gates Monitoring gates Undefined gates Total

Commercial feasibility 10 3 6 19Technical feasibility 8 3 5 16Strategic fit 3 3 5 11Budget feasibility 8 1 1 10Market feasibility 2 4 1 7Manufacturing feasibility 3 1 2 6Operational feasibility 3 1 4Registration feasibility 2 1 3Clinical trial feasibility 3 3Vaccination feasibility 2 2Launch feasibility 1 1Total number of gates 12 5 6


13

257

The most prominent evaluation criteria are commercial feasibility, technical feasibility, strategic fit and budget feasibility. Throughout the value chain, the sub-criteria per criteria may evolve. Additionally, while progressing through the value chain the criteria are more strictly applied. A detailed overview of the internal criteria per gate are shown in Supplementary Material S13.4 (defined gates), S13.5 (monitoring gates) and S13.6 (undefined gates).

A gap-ovelap analysis showed that almost half of the internal criteria are directly aligned with external criteria (see Figure 13.6). These criteria ensure that the vaccine candidate progresses through the external checkpoints. Six internal criteria are unmatched with external criteria. These criteria ensure the long-term viability of the stakeholders who develop the vaccine candidates.

Societal feasibility

Adequate pricing

Adequate relevance / unmet need

Market feasibility

Adequate product performance Technical feasibility

Adequate clinical trial design Clinical trial feasibility

Adequate quality

Adequate supply systemManufacturing feasibility

Adequate operational performance Operational feasibility

Commercial feasibility

Budget feasibility

Strategic fit

Vaccination feasibility

Registration feasibility

Launch feasibility

Match

edcrite

riaUnm

atch

edcrite

ria

External criteria Internal criteria

FIGURE 13.6 | Five internal criteria are matched with external criteria; six internal criteria are unmatched but ensure the long-term viability of the stakeholders who develop the vaccine candidates.

Chapter 13

13

258

13.4 Discussion

Here we find 29 distinct stages and 28 corresponding gates that describe the vaccine innovation process in ten distinct but integrated workstreams. Next to the conventional research and development stage-gates of vaccine development, this study provides stage-gates for nine additional types of activities that are essential for successful vaccine innovation. The cyclical nature of the VIC highlights the importance of synchronizing with unmet needs, market changes, existing expertise and quality improvements. Additionally, it conceptualizes the difference between incremental and radical vaccine innovation. By highlighting the gap between internal criteria used to evaluate the progress of vaccine candidates and external criteria applied by competent authorities, newcomers to the field can enhance their viability.

The current study comprehensively describes the activities and criteria in vaccine development per stage-gate as perceived by KOLs. Existing studies mainly describe the conventional and defined stage-gates of vaccine R&D and much focus is placed on the complexity of manufacturing (581, 592-594). In addition, detailed descriptions exist for other technical steps within the VIC, including fundamental vaccine research (595), clinical vaccine development (596), and vaccination strategies (597). The current study therefore adds to the existing body of knowledge by extending the description of vaccine innovation to include less defined processes such as funding and business development, post-marketing surveillance, and other monitoring processes.

Importantly, the VIC highlights that whereas some activities in vaccine innovation take place in clearly defined and sequential workstreams (defined stages and gates), the occurrence and timing of other activities is contingent on contextual factors (undefined stages and gates) and yet other activities take place continuously within a certain range of activities (monitoring stages and gates). Moreover, the VIC depicts how these sub processes are executed in parallel in order to reduce development timelines. Such parallel processing is a necessary approach to reduce delays to authorization, especially in the context of public health emergencies (568).

The cyclical nature of the model clarifies that research can be executed through technology push (with a starting point in the exploration & discovery domain) or through market pull (via synchronisation of research with unmet societal needs) (563). In the latter case, the societal relevance of explorative research may be optimised (598). As an example, a recent study argued that differential articulations of medical, societal and technical unmet needs were likely to result in mismatches in resource allocation and delays in adequate responses to the 2014 Ebola outbreaks (599). In contrast, a thorough understanding and concordance of unmet needs can align actors involved in innovation and accelerate responses to (emerging) infectious diseases (219).

The circular nature of the model additionally enables the conceptual distinction between incremental and radical innovations. Most new vaccines are incremental innovations and therefore part of a circular value chain for similar product-market combinations. Examples of such incremental innovations include the generation of influenza vaccines against novel strains (595, 600), and changes to the intended use or manufacturing process of existing vaccines (581). In some other cases, development of novel vaccines is accelerated due to the identification of correlates


13

259

of protection in the clinical setting. All these incremental innovations can be conceptualized as new iterations of the vaccine innovation cycle for the same product-market combination in which at least a number of stages are informed by successful completion of the VIC by previous vaccine products (581). The cyclical nature thus also takes into account the comfortable position of existing players that build upon previous iterations of the VIC and the market introgression difficulties that newcomers face.

In contrast to incremental innovations, radical innovations create novel product-market combinations and innovators can determine their own starting point in the cycle. From there they continue their innovation efforts throughout the cycle. Innovations may originate from a clinical setting, industrial process optimization may contribute to fundamental knowledge creation and through transdisciplinary research, scientists and clinicians can simultaneously develop and implement knowledge. This is conceptualized through the lack of a clear starting point in the cycle; innovations can start anywhere and from there continue their way through the innovation stage-gates, as was shown in the Cyclic Innovation Model (518).

The lack of a clear starting point facilitates interpretation of the model from different stakeholder perspectives. As mentioned by one of the respondents from the survey: “[Where you start] all depends from whose stakeholder perspective you look at the innovation cycle - as an innovator, national policy maker, manufacturer, donor, investor etc.” Additionally, it is important to point out that many elements depicted at the stage level are in essence non-linear as well. These include activities that imply iteration, evaluation, and feedback and feed-forward (e.g. in exploration and discovery, CMC and market preparation), and demonstrate that innovation is non-linear at heart (518).

By investigating and explicating the internal and external criteria that are used in different stages of vaccine innovation, the current study contributes to improved stakeholder understanding of the constraints and requirements for vaccine innovation. Out of the five most prevalent criteria employed by vaccine innovators, only two are linked to criteria employed by competent authorities. This emphasizes the need for the vaccine industry to monitor their ‘bottom line’ and long-term viability before investing resources in developing vaccines that meet external unmet needs (573). This is especially relevant for SMEs who struggle to acquire the relevant expertise and funding to bring vaccine candidates closer to market deployment (574). This description of internal criteria may improve reciprocal understanding. By acting upon this understanding to create win-win situations, stakeholders can improve their collaboration with other players, as previously described for Ebola (599).

Interestingly, market feasibility was not mentioned as a criterion in later stage-gates within R&D processes. Moreover, the operationalization of market feasibility by the KOLs in our study was less inclusive than the definition used in the Total Systems Effectiveness framework that is currently being developed by the WHO to assist procurement stakeholders in decision-making (601). This discrepancy may in part be explained by our decision to focus on criteria for vaccines for developed countries, where constraints to market feasibility are less prominent. In contrast, market feasibility is a step-limiting factor for vaccines in developing countries, as elaborated on for disease-specific, not-for-profit product development partnerships (602). Differences in criteria between vaccines for developed and developing countries are also prominent in the

Chapter 13

13

260

societal domain. As shown before for Ebola (219, 599) and rabies (298, 603), vaccine innovation for developing countries is primarily guided by public perception and political prioritization. In line with these findings, the KOLs in the current study commented that the influence of political and financial factors depends on the visibility, perception and urgency of the disease in question.

The execution of stage-gates, the timing of parallel running workstreams, and the rigidity with which criteria are applied are dependent on contextual heterogeneities. For instance, depending on the context, outcomes from monitoring gates may include project adjustments rather than strict go / no-go outcomes. In industry, the CMC stage is generally completed before late-stage preclinical studies are started, whereas academic organizations often continue into late-stage preclinical studies before ascertaining the feasibility of large-scale production under GMP conditions. This may result in unnecessary delays since late-stage preclinical studies may need to be reiterated contingent on changes in the production process (568, 577). In this sense, the VIC is a descriptive model, describing which stage-gates are seen as vital aspects of vaccine innovation, rather than a prescriptive model that determines which criteria should be taken into account in an ideal world.

13.5 Conclusion

This study provides an overview of how different processes align and together contribute to successful vaccine innovation. The integrated, actor- and domain-transcending perspective of the VIC provides a comprehensive view on the necessary stages and relevant criteria for successful progression through the vaccine value chain. The resulting model can be used by all key stakeholders to evaluate how their activities relate to those of others and to facilitate cross-learning, understanding and appreciation among stakeholders. Such a boundary-spanning function may contribute to reducing innovation process inefficiencies and additionally inform stakeholders about product development timelines, costs and risks, thereby helping with better prioritization and allocation of resources. Considering the different levels of understanding of the total value cycle, herewith we hope to improve both understanding and communication in forming collaborative alliances and partnerships between public and private partners to accelerate vaccine innovation processes.

13.6 Acknowledgements

The authors gratefully acknowledge the assistance of Tommy Riemens MSc and Matthijs van der Linde MSc in data collection and analysis. This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.


13

261

towards improved process efficiency in vaccine innovation ... 13 - towards improved... · towar ove...

Documents