i-racm: a fully integrated risk and life cycle cost model · framework, providing for a seamless...
TRANSCRIPT
American Institute of Aeronautics and Astronautics
1
I-RaCM: A Fully Integrated Risk and Life Cycle Cost Model
Dominic DePasquale1 and A.C. Charania2
SpaceWorks Engineering, Inc. (SEI), Atlanta, GA, 30338
SpaceWorks Engineering, a business division of SpaceWorks Engineering, Inc. (SEI),
conceived of the Integrated Risk and Cost Model (I-RaCM) to meet the need for
synchronous cost and risk assessment early in the design of a new system. I-RaCM
encompasses a diverse set of cost and risk tools implemented in the ModelCenter®
framework, providing for a seamless integrated analysis solution. Both newly developed and
existing industry-standard software tools are combined within I-RaCM allowing for rapid
evaluation of life cycle cost, operations, reliability, technology development costs, and
commercial business case viability. Tools currently integrated within I-RaCM include the
NASA Air Force Cost Model (NAFCOM), Galorath SEER-H, SpaceWorks Engineering’s
Facilities and GSE Operations Assessment (FGOA) Tool, and a reliability calculation tool
using Fault Trees and Event Sequence Diagrams. Also included is an initial version of a new
Technology Cost Estimation (TCE) tool developed to estimate technology investment and
development costs to achieve TRL-6 status based on a simple set of relevant inputs. Finally,
the I-RaCM platform contains a custom created system level cost/risk aggregation tool,
called Stack’em, which collects, post-processes, and graphically summarizes key outputs.
Several novel cost visualization techniques have been conceptualized for Stack’em, providing
designers with comprehensive insight into major cost and risk outputs, along with a budget
optimization process that automatically adjusts program expenditures and schedule to fit
under a fixed budget curve.
This paper details the current functionality and toolset of I-RaCM. Results from four
case studies demonstrating the capabilities of I-RaCM through analysis of a notional
present-day NASA lunar exploration architecture are also presented. The entire integrated
environment of I-RaCM is capable of linking with performance disciplinary tools through
pre-established input and feedback interfaces. SpaceWorks Engineering plans to continue
expanding I-RaCM with the addition of more advanced reliability analysis, availability and
performability tools, and discrete event simulation of operations processing.
Nomenclature
ABM = Agent Based Modeling LCC = Life Cycle Cost
CABAM = Cost and Business Analysis Module LEO = Low Earth Orbit
CER = Cost Estimating Relationship LLO = Low Lunar Orbit
CEV = Crew Exploration Vehicle LOC = Loss of Crew
CaLV = Cargo Launch Vehicle LOM = Loss of Mission
CLV = Crew Launch Vehicle LOR = Lunar Orbit Rendezvous
CM = Command Module LSAM = Lunar Surface Access Module
DD = Design & Development MS = Microsoft
DDT&E = Design, Development, Testing & Evaluation NAFCOM = NASA Air Force Cost Model
EDS = Earth Departure Stage NASA = National Aeronautics and Space Administration
ESAS = Exploration Systems Architecture Study NESC = Nodal Economic Space Commerce [Tool]
ESD = Event Sequence Diagram SM = Service Module
EVA = Extra-Vehicular Activity T&E = Testing and Evaluation
1 Director, Engineering Economics Group, SpaceWorks Engineering, 1200 Ashwood Parkway - Ste. 506, AIAA Member.
2 President, SpaceWorks Commercial, 1200 Ashwood Parkway - Ste. 506, AIAA Senior Member.
American Institute of Aeronautics and Astronautics
2
FGOA = Facilities and GSE Assessment [Tool] TEI = Trans-Earth Injection
FTA = Fault Tree Analysis TIM = Technology Impact Matrix
GSE = Ground Support Equipment TFU = Theoretical First Unit [Cost]
I-RaCM = Integrated Risk and Cost Model TLI = Trans-Lunar Injection
ISS = International Space Station TPS = Thermal Protection System
IVHM = Integrated Vehicle Health Monitoring TRL = Technology Readiness Level
I. Introduction
IABILITY of any aerospace program in the present era of scarce resources and competing national objectives
is ultimately determined by its life-cycle economics. Successful analysis and design of a new system must
therefore balance cost, risk, and performance to develop a product that satisfies customer objectives and is robust to
uncertainty. It is advantageous to fully assess these interrelated metrics concurrently during the conceptual design
phase when the greatest design freedom is available; however performance is typically considered independently of
cost and risk in the traditional early design process. A greater pressure on program managers to deliver more capable
products on schedule and within the available budget has elevated the importance of cost and risk driven design, but
integration into the existing systems engineering process is considerably challenging. The Integrated Risk and Cost
Model (I-RaCM) meets these challenges and delivers the advantages that only a seamless integrated solution of cost
and risk models can provide.
A. Motivation for an Integrated Risk and Cost Tool The ability to understand and accurately forecast the cost and risk associated with major aerospace programs has
an important and expanding role as program managers face increased pressure from commercial, societal, and
political sources. Tightening budgetary constraints and reduced tolerance for cost overruns necessitate accurate
lifecycle cost estimates by program managers and designers. Political pressure and increased public scrutiny also
encourage program managers to communicate realistic estimates of cost and risk associated with a project.
Furthermore, it is often advantageous to make compromises between system performance and cost/risk to best meet
mission objectives. Finally, the movement toward shorter design cycles and the need for immediate decision-making
early in the design process requires that cost and risk insight is quickly and accurately generated and analyzed.
While the need for assessment of cost and risk may be recognized, accurately modeling these complicated
metrics during conceptual design presents several challenges. The addition of cost and risk parameters increases the
size of an already large trade space, and time constraints can prevent full exploration of design options. It is also
difficult to calculate and quantify metrics such as affordability, reliability, and other “ilities” due to a lack of data or
experience. Analyzing sensitive programmatic factors like cost can also expose the design team to additional
scrutiny and concern if the initial cost estimates, prior to optimization, are high relative to the budget.
B. Advantages to an Integrated Risk and Cost Tool
Success with multi-disciplinary performance analysis frameworks has demonstrated the value of integrated tool
sets during conceptual design. SpaceWorks Engineering in particular has previously used integrated performance
models to conceptually design reusable launch vehicles and other applications.1,2,3,4
In addition, SpaceWorks
Engineering has applied multi-disciplinary design approaches to rapidly assess technology investment to various
aerospace systems.5,6,7,8
Cost, reliability, and other “ilities” have been included in these multi-disciplinary
frameworks to a limited extent, providing a foundation in thinking for a fully integrated toolset. I-RaCM offers a
more complete set of “ility” analysis tools, many with higher fidelity and greater flexibility than previously
integrated together. Specific advantages of I-RaCM over current practices include:
1. Complete Cost Picture: I-RaCM provides a more complete picture of cost and risk associated with a project
over its life cycle. Design, Development, Testing, and Evaluation (DDT&E) and Theoretical First Unit (TFU) costs
are common to cost estimates, but often the fixed and recurring costs of operations are neglected. The cost of
failures and the cost of risk reduction are even less frequently accounted for in early cost estimates. Through its
contained toolset, I-RaCM accounts for all major conceivable life cycle costs, and includes risk in the analysis loop.
2. Utilization of Existing Best-of-Class Tools: The integration of multiple tools has several advantages as
opposed to the creation of a single do-all cost and risk application. The use of an integrated tool set results in more
meaningful analysis by allowing for individual best-in-class tools to perform the particular function for which they
are most suited. The availability of readily integrated individual tools also allows for the user to select a specific
V
American Institute of Aeronautics and Astronautics
3
subset of I-RaCM tools to best analyze a particular problem. Upgrades to the integrated model are also easily
accomplished by upgrading the individual tools, and the integrated model maintains its integrity as long as the input
and output links between individual tools remain the same. By comparison, compromises in capability that one
makes to achieve a single do-all application may result in diluting the effectiveness and flexibility of the tool.
3. Interface with Other Design Disciplines: Inputs are often fed to cost analysis tools as the last step in a
traditional, performance oriented, design process. Including cost and risk not just as an afterthought, but as an active
discipline in the design process, can greatly improve the cost to benefit ratio of the final design. I-RaCM will have
the ability to easily interface with performance disciplines implemented in Phoenix Integration’s ModelCenter®
integration framework.
4. Analytical Consistency: A serial human-in-the-loop process for cost and risk analysis is prone to errors in
communication and data entry. An integrated cost and risk process, with established and verified data transfer
protocols between inputs, is more likely to maintain analytical consistency and eliminate transcription errors.
5. Efficiency: An automated tool is more efficient than a traditional human-in-the-loop serial design process.
Changes at any level in the design can rapidly propagate through the cost tools without any need for human
intervention. Design trades can then be automatically assessed in a manner that includes system optimization and
uncertainty assessment. The automated nature of the tools allows for the thousands of iterations necessary for these
tasks to be rapidly accomplished.
6. Investigation of Uncertainty: Sources of risk can be more thoroughly understood with the capability to assess
uncertainty in cost, reliability, and schedule. By defining distributions for key input variables, the uncertainty
associated with outputs can be investigated using probabilistic Monte Carlo analysis. I-RaCM has the ability to
conduct probabilistic uncertainty analysis using PiBlue Software or ModelCenter® plug-in drivers.
7. Optimization: The ability to interface with other design disciplines and the collection of design tools within
one framework allows for design optimization. I-RaCM can utilize a suite of industry standard optimization
algorithms to demonstrate optimization with low cost or risk as an objective function.
II. I-RaCM Overview and Tools
I-RaCM encompasses a diverse set of cost and risk tools implemented in the Phoenix Integration ModelCenter®
framework, providing for a seamless integrated solution. Both newly developed and existing industry-standard
software tools are combined within I-RaCM, allowing for rapid evaluation of life cycle cost, operations, reliability,
technology development costs, and business case viability. ModelCenter® provides the means to link multiple
software tools together, control the tools at runtime, and performs data collection and processing. Development of I-
RaCM is ongoing to expand the comprehensiveness of its cost and risk analysis capabilities. SpaceWorks
Engineering plans to add more advanced reliability analysis, availability calculation, and discrete event simulation
of operations processing. The graphic of Figure 1 depicts the current and planned disciplinary analysis tools within
I-RaCM. The integration of tools denoted as partially current and partially future is in progress. The initial ability to
operate these tools in the ModelCenter® environment has been demonstrated, but they are either not yet fully
developed or not yet fully tailored for I-RaCM.
I-RaCM contains several tools and capabilities that are first-of-their-kind accomplishments for an integrated
modeling environment and the conceptual deign community. The integration of the government-sponsored NASA
Air Force Cost Model (NAFCOM) and Galorath’s SEER-H hardware cost model into I-RaCM is of particular
consequence and value. These two tools are commonly used for cost estimating in the aerospace industry, and the
authors are unaware of any previous integrated model containing their full versions. SpaceWorks Software’s
commercially available Remix software tool allows for the full, GUI-executed, versions of NAFCOM and SEER-H
to interface with ModelCenter®. Another unique capability included is an initial version of a new Technology Cost
Estimation (TCE) tool developed to estimate technology investment and development costs to achieve Technology
Readiness Level (TRL) 6 status based on a simple set of relevant inputs. The I-RaCM platform also contains a
custom created system level cost/risk aggregation tool (Stack’em) which collects, post-processes, and graphically
summarizes key outputs. Several novel cost and visualization techniques have been conceptualized for Stack’em,
providing designers with comprehensive insight into the major cost and risk outputs. Stack’em also contains a a
American Institute of Aeronautics and Astronautics
4
budget optimization capability that automatically adjusts program expenditures and schedule to fit under a fixed
budget curve.
In addition to disciplinary analysis tools in I-RaCM, other tools are available to define program life cycle inputs
and conduct probabilistic simulation. The Campaign Definition Spreadsheet allows the user to lay out a year-by-year
mission profile for all elements of the architecture or system. The Technology Impact Matrix (TIM) allocates the
impacts of selected technologies amongst the applicable architecture elements. Together with TCE, the TIM
provides the capability for I-RaCM to assess the development cost of technologies themselves as well as the
development and production cost implications of the element to which the technology is applied. Finally, the
ProbWorks© and OptWorks© driver tools allow for probabilistic analysis of the entire integrated model and global
optimization respectively. Other tools exist to perform probabilistic analysis and optimization within ModelCenter®
and these could be easily substituted if preferred.
The complete set of the tools currently integrated within I-RaCM, as well as those planned for inclusion in the
future, are listed in Table 1. While all of the I-RaCM tools listed here are available, not all must be used when
conducting an analysis. I-RaCM provides the capability to easily link any of these tools, but instances of I-RaCM
may exist with only a subset of the available tools, depending on the problem being analyzed. Detailed descriptions
of each of the current and partially integrated tools listed in Table 1 are given in the following sections.
NAFCOM
SEER-H
Development and Production Cost
NAFCOM
SEER-H
Development and Production Cost
Facilities and Ground Ops Analysis (FGOA)
Facilities and Operations Cost
Facilities and Ground Ops Analysis (FGOA)
Facilities and Operations Cost
Cost And Business Analysis Module (CABAM)
Nodal Economic Space Commerce (NESC) Tool
Business Case Evaluation
Cost And Business Analysis Module (CABAM)
Nodal Economic Space Commerce (NESC) Tool
Business Case Evaluation
Event Sequence Diagrams
Fault Trees
Reliability
Event Sequence Diagrams
Fault Trees
Reliability
Technology Cost Estimator (TCE)
Technology Cost
Technology Cost Estimator (TCE)
Technology Cost
Descartes Discrete Event Simulation in Arena
Operations Scheduling and Simulation
Descartes Discrete Event Simulation in Arena
Operations Scheduling and Simulation
Probabilistic Risk Assessment
Risk / Reliability and Consequences
Probabilistic Risk Assessment
Risk / Reliability and Consequences
Stochastic Petri Nets
Availability and Performability
Stochastic Petri Nets
Availability and Performability
Data Handlingand Integration
Stack’em
Output Visualization and Exploration
Stack’em
Output Visualization and Exploration
I-RaCM: Integrated Risk and Cost Model Legend:
<Preferred Tool>
<Function>
<Preferred Tool>
<Function>
Current
Future
PHX Integration
ModelCenter
Figure 1: I-RaCM Disciplinary Analysis Tools
American Institute of Aeronautics and Astronautics
5
A. PHX ModelCenter® Integration Environment
Tool Name: ModelCenter®
Function: Software Tool Integration
Platform: Windows® Executable
Origination: Developed and commercially sold by Phoenix Integration, Inc.
Description: ModelCenter® allows for the coupling of various disciplinary software tools into one integrated
design model. ModelCenter® provides a user-friendly graphical front-end and various built-in
capability for linking the tools together (feed-forward and feed-back links), performing trade
studies, and collecting and analyzing data from a large number of executions of the integrated
software tools. The interface between ModelCenter® and each software tool is defined by a
wrapper file. The wrapper file contains code instructing ModelCenter® how to transfer
input/outputs and execute the integrated software tool. A wrapper was created for each of the cost
and risk tools contained within I-RaCM. Creation of a wrapper is generally time intensive, so for
many of the tools rather than create a wrapper specific to a fixed set of inputs and outputs, code
was written to automatically generate a wrapper for many alternative instances of the tools. This
approach greatly simplifies setup of I-RaCM essentially allowing for an integrated model to be
easily created for any application that the toolset is capable of analyzing.
B. NASA Air Force Cost Model (NAFCOM) and Remix
Tool Name: NASA Air Force Cost Model (NAFCOM)
Function: Non-Recurring Development Costs and Recurring Production Costs
Platform: Windows® or Macintosh® Executable
Origination: NAFCOM is a government-sponsored cost model with copyright held by Science Applications
International Corporation (SAIC)
Description: NAFCOM estimates non-recurring costs including Design, Development, Testing, and Evaluation
(DDT&E) and Theoretical First Unit (TFU) costs by employing Cost Estimating relationships
(CERs) derived from the actual costs of various historical systems. Several CER methodologies
are available and required inputs can include common weight breakdown structure masses,
programmatic complexities, and subsystem-specific variables (duration, volume, electrical power,
structural efficiency, etc.). Programmatic costs are also estimated including system test hardware,
Table 1: Complete List of I-RaCM Tools
I-RaCM Function Tool(s) / Methodology
CURRENT TOOL SET
Integration Environment ModelCenter® by Phoenix Integration
Nonrecurring Cost (DDT&E and TFU) NAFCOM via the Remix Wrapper Generator
SEER-H by Galorath, Inc. via the Remix Wrapper Generator
Facilities and Equipment Cost Facilities and GSE Operations Assessment (FGOA) Tool
Reliability Reliability_Calc Tool Event Sequence Diagrams and Fault Trees
Campaign Profile Definition Campaign Manager
Technology Effects Technology Impact Matrix
Output visualization and exploration Stack’em
Probabilistic Design Analysis ProbWorks by PiBlue Software
Optimization OptWorks by PiBlue Software
PARTIALLY INTEGRATED TOOLS
Technology Investment Technology Cost Estimator (TCE) Prototype
Operations Simulation Descartes Discrete Event Simulation in Arena®
Business Case and Market Evaluation Cost and Business Analysis Module (CABAM)
Nodal Economic Space Commerce Tool (NESC)
FUTURE TOOLS
Risk/reliability and Consequences Probabilistic Risk Assessment
Availability and Performability Stochastic Petri Nets, Markov Models
Analysis Data Sharing Portable Visualizer
American Institute of Aeronautics and Astronautics
6
integration, assembly, & checkout, system test operations, ground support equipment, systems
engineering & integration, and program management. The Remix software, described below,
provides a means for ModelCenter® to interface with the GUI-only NAFCOM.
Tool Name: Remix
Function: Interface NAFCOM with ModelCenter®
Platform: Windows® Executable
Origination: Developed by SpaceWorks Engineering and commercially marketed by SpaceWorks Software
Description: Remix provides a means for ModelCenter® to interface with the GUI-only NAFCOM by
automatically generating a wrapper file for any unique NAFCOM model and by controlling the
NAFCOM GUI at runtime. NAFCOM does not have command line or simple text file run modes,
so the GUI controller effectively “clicks” on the controls to execute the necessary functions of the
program. The current version of Remix is capable of generating wrappers that provide
ModelCenter® interfaces for all conventional cost estimating equation (CER) input variables,
conventional CER complexity factor inputs, complexity generator (CG) CER universal factors,
and CG subsystem-specific input variables (e.g. structural efficiency, design lifetime, volume,
output power). In addition, Remix-generated wrappers include the Liquid Rocket Engine Cost
Module (LRECM) submodel to NAFCOM. LRECM was developed by Boeing’s Rocketdyne
division (now Pratt & Whitney Rocketdyne) and provides an alternative liquid rocket engine cost
estimating methodology with input variables such as thrust, chamber pressure, propellant type,
engine cycle, and process improvement factors. Systems Integration input variables are not yet
included in Remix. Output variables (DDT&E, STH, FU, DD, and Production cost) for all
subsystems and Systems Integration are included. Remix takes only a matter of seconds to
generate wrappers for NAFCOM models to be run either with or without “Risk,” NAFCOM’s
terminology for uncertainty.
C. SEER-H and Remix Tool Name: SEER-H
Function: Non-Recurring Development Costs, Recurring Production Costs, Operations and Support Costs
Platform: Windows® Executable
Origination: SEER-H is a commercial software product available from Galorath, Inc.
Description: SEER-H estimates the costs of development, production, operations, and support for mechanical,
electronic, structural, and hydraulic hardware elements. CERs underlying the cost estimates are
derived from the actual costs of various hardware components. Knowledge Bases are available to
select appropriate cost analogies for each system component and to adjust variables defining
mission, program, development environment, and production environment cost factors. Cost
outputs can be obtained either as point value estimates or probabilistically. Programmatic costs are
also estimated including system test hardware, integration, assembly, & checkout, system test
operations, ground support equipment, systems engineering & integration, and program
management. The Remix software, described below, provides a means for ModelCenter® to
interface with SEER-H.
Tool Name: Remix
Function: Interface SEER-H with ModelCenter®
Platform: Windows® Executable
Origination: Developed by SpaceWorks Engineering and commercially marketed by SpaceWorks Software
Description: Remix provides a means for ModelCenter® to interface with SEER-H by automatically generating
a wrapper file for any unique SEER-H model. Remix takes advantage of SEER-H’s Server Mode
to execute SEER-H in conjunction with a semaphore and command file. The current version of
Remix is capable of generating wrappers that provide ModelCenter® interfaces for nearly all
SEER-H project, roll-up, mechanical work element, electronic work element, operations and
support, site, and add-in parameters. System Level Analysis parameters are also included in the
wrapper generated by Remix. Knowledge base and SEER-H calculated values for inputs are
preserved by the wrapper. Output variables (DDT&E, Production, Operations, and other costs) for
all subsystems and Systems Integration are included. Remix takes only a matter of seconds to
generate wrappers for SEER-H models to be run either deterministically or probabilistically.
American Institute of Aeronautics and Astronautics
7
D. Facilities and Ground Operations Assessment (FGOA) Tool
Tool Name: Facilities and Ground Support Equipment Operations Assessment (FGOA)
Function: Ground and Infrastructure Costs
Platform: MS Excel®
Origination: FGOA was previously developed by SpaceWorks Engineering with assistance from Construction
Cost Consultants, Incorporated in 2004 under contract to NASA Kennedy Space Center
Description: FGOA is a spreadsheet tool used to calculate and aggregate launch vehicle infrastructure costs and
inflate them to desired year dollars. NASA KSC periodically publishes its Real Property Database
and other technical databases of GSE costs. These databases are used to update and calibrate
FGOA. SpaceWorks Engineering adjusts the estimates as necessary to account for variances from
the historical estimates (e.g. unusual costs of copper wiring in today’s market, variances in the
costs of steel and supplies, new regulations and requirements, changes in labor costs).
E. Reliability_Calc
Tool Name: Reliability_Calc
Function: Reliability Estimation
Platform: MS Excel®
Origination: SpaceWorks Engineering internally developed reliability analysis tool
Description: Reliability_Calc uses a combination of Fault Tree Analysis (FTA) and Event Sequence Diagrams
(ESD) to quantify top-level reliability metrics such as Loss of Mission (LOM) and Loss of Crew
(LOC) for the complete system or architecture being modeled. FTA is used to find failure rates of
given designs based on a bottoms-up reliability failure analysis. For an architecture analysis, an
FTA is developed for each transportation element of the architecture. These individual FTA results
are then combined into an overall architecture level FTA to determine loss of mission results. The
component failure rates determined by FTA are also combined into an ESD which determines the
overall architecture crew survival rates. The ESD uses the results from several FTAs as well as
success rates associated with various mitigation scenarios in order to determine crew survival.
Monte Carlo analysis is also integrated into the reliability assessment tool to provide mission
success and crew survival rates that meet a given certainty level. Reliability_Calc contains a
macro for the automatic generation of a ModelCenter® wrapper for this tool such that it can be
integrated within I-RaCM. This automatic wrapper generation capability eases the burden on the
user when new elements or events are added to the FTA. The user can simply run the macro to
generate a wrapper for the current set of inputs and outputs, regardless of the system for which the
fault trees and event sequence diagrams have been customized.
F. Technology Cost Estimator (TCE)
Tool Name: Technology Cost Estimator (TCE)
Function: Early Technology Development Costs
Platform: MS Excel®
Origination: A prototype of this tool has been developed by SpaceWorks Engineering
Description: TCE provides estimates for new technology development by relating a simple set of inputs to an
underlying database of historical technologies. The model is designed to estimate the cost for each
Technology Readiness Level (TRL) step-change (e.g. from 1 to 2 or 4 to 5) up to a TRL of 6. A
concise set of qualitative and quantitative user inputs such as current TRL, length of the research
activity, degree of funding availability, and extent of revolutionary innovation define the
characteristics of the candidate technology. The cost estimate is made possible in part by a
database of historical technology development costs. The technologies in the database ranging
across several technology categories including avionics, manufacturing, operations, propulsion,
structure/materials, subsystems, and thermal protection systems (TPS) may be selected by the user
for inclusion or exclusion in the estimating equation. The current model is in beta form (version
0.6) and has the general qualities of the final model along with a database of approximately forty
technologies in the database.
American Institute of Aeronautics and Astronautics
8
G. Campaign Manager
Tool Name: Campaign Manager
Function: Define the Desired Missions to be Conducted Over the Campaign Lifetime
Platform: MS Excel®
Origination: This tool was developed by SpaceWorks Engineering
Description: The campaign manager allows the user to define the desired mission schedule for various elements
of an aerospace architecture. The campaign start year and number of elements to be utilized in
each year of the campaign are input in tabular form. The campaign can span up to 16 years in the
current version of the campaign manager. A ModelCenter® wrapper was also created for the
campaign manager for its integration into I-RaCM. The I-RaCM user can easily evaluate the
effects of various campaigns on life cycle cost and reliability by inputting different mission
schedules.
H. Technology Impact Matrix
Tool Name: Technology Impact Matrix
Function: Allocate Impacts of Developed Technologies Amongst Inputs to Other Tools
Platform: MS Excel®
Origination: This tool was developed by SpaceWorks Engineering
Description: A technology impact matrix (TIM) is a matrix of available technologies versus relevant input
variables of other tools in the integrated model. The matrix is populated with the absolute setting
or multiplicative factor for those input values which are affected when each technology is active or
inactive. The technology impacts captured in the TIM have some positive or negative effect on
performance, cost, or reliability metrics. The Technology Impact Matrix maps the use of a given
technology to its impact on other design variables such that these metrics can be quantified.
Application of a Technology Impact Matrix in the I-RaCM environment has the potential to
provide a more complete picture of the true cost versus benefit of employing a particular
technology. In a typical conceptual design environment, a TIM may be used to appropriately
reflect the impact of technologies on performance values. The development cost of these
technologies may also be taken into account, but the use of a technology may also have cost
implications for the development and production of the element to which the technology is
applied. For example, the use of composite propellant tanks has the advantage of lower tank mass,
but may also have implications for manufacturing and design integration. The use of a TIM in a
multidisciplinary framework like I-RaCM allows for these primary and secondary aspects of
technology development to be taken into account.
I. Stack’em
Tool Name: Stack’em
Function: Program Phasing, Output Visualization and Exploration
Platform: MS Excel®
Origination: This tool was developed by SpaceWorks Engineering
Description: The purpose of Stack’em is to collect, post-processes, and graphically summarize key outputs
from the upstream analyses in I-RaCM. Cost, reliability, and risk (uncertainty distribution
parameters) generated by NAFCOM, FGOA, TCE, and other tools are input to Stack’em where
they are evaluated over the program life cycle. Design and Development (DD) costs are spread
over time using user input variables defining beta curves, DD phase duration, pre-phase A cost as
a percentage of DD cost, pre-phase A duration, and sustaining engineering cost as a percentage of
DD cost. System Test Hardware and up to five planned tests can be defined to calculate Test and
Evaluation (T&E) costs. Production costs over the life of the program take into account learning
curve effects, unit production rates, and the campaign mission model. The user can also input
program budget and overhead wraps data such as projected budget values by year, a percentage
above total cost to hold as reserves, a number of years to shift this reserve if desired, and the
percentage of total cost to charge to the program for full cost wraps. Visual graphics produced by
Stack’em aid the user in rapidly evaluating the outputs of I-RaCM and in performing trade studies.
A “sand chart” depicting the cost roll-up is provided, along with other charts showing costs by
category (DD, T&E, production, and technology maturation), a Gantt chart style timeline for these
categories, two reliability breakdown charts, and several others. Stack’em has built-in trade space
American Institute of Aeronautics and Astronautics
9
exploration features such as the ability to optimize program schedule in order to meet budget
constraints. A ModelCenter® wrapper necessarily complements Stack’em for integration with the
other tools, and the Stack’em tool has a macro that can automatically generate this wrapper.
J. Descartes
Tool Name: Descartes9
Function: Operations Simulation
Platform: Windows® Executable combined with MS Excel®
Origination: Descartes was developed by SpaceWorks Engineering and is implemented within Rockwell
Automation’s commercially available Arena® software
Description: Descartes is a Discrete Event Simulation (DES) approach to operations analysis of aerospace
systems in Rockwell Automation’s Arena® software. Descartes can be employed to simulate the
operations of applications such as lunar surface systems, launch vehicles, and in-space supply
chains. Descartes-Hyperport is one implementation of Descartes currently in development for the
analysis of advanced reusable launch vehicle operations. Microsoft Excel® and VBA interfaces
enable integration of Descartes-Hyperport into I-RaCM. Inputs for the analysis model include
descriptions of the basic vehicle concept of operations, mission models, and performance
information. Typical outputs from Descrates include turn-around time, operations costs, number
and specialties of personnel, and equipment needs.
K. Nodal Economic Space Commerce (NESC) Tool
Tool Name: Nodal Economic Space Commerce (NESC) Tool10,11,12
Function: Commercial Business Case Evaluation and Market Simulation
Platform: Java Executable
Origination: Developed by SpaceWorks Engineering
Description: The NESC model is a dynamic space market simulation and financial engineering tool that uses
Agent-Based Modeling (ABM) techniques to simulate the complex interactions between supply
and demand. ABM has been used in the past to represent plants and animals in ecosystems,
vehicles in traffic, and autonomous characters in animation and games. A wide range of
organizations including Macy's, Humana, and the U.S. Air Force Space Command have used and
benefited from ABM simulations. Agent-based economic simulation is of higher fidelity than
spreadsheet models, and better reflects reality by expressly modeling the individual actions and
interactions of companies, their customers, and their competitors. NESC enables the simulation of
price competition, new entrants to the market, reliability effects, and product differentiation.
Various types of NESC model inputs (required economic return, vehicle capability, costs, etc.)
define the worldview and behaviors of entities in the model. Each company “agent” autonomously
decides its pricing strategy given market conditions and limited competitor information, together
with its own unique costs and product characteristics. Exercising NESC probabilistically is a
necessity since each run results in a different outcome depending on the decisions of the agents.
Probabilistic simulation also allows for investigation of reliability effects. NESC outputs the
financial health of each company (cash flows, Net Present Value, market share, etc.) and can be
used to explore various scenarios including supply vs. demand effects, customer influences, price
changes over time, and product differentiation. NESC has previously been employed by
SpaceWorks Engineering to analyze the International Space Station transportation services market
and the sub-orbital space tourism market.
L. Cost and Business Analysis Module (CABAM)
Tool Name: Cost and Business Analysis Module (CABAM)13
Function: Commercial Business Case Evaluation
Platform: Excel®
Origination: Initially developed at the Georgia Institute of Technology then further developed and modified by
SpaceWorks Engineering
Description: CABAM determines the cash flows and other financial measures of success for a hypothetical
company given programmatic, cost, and market demand inputs. Program definition inputs include
fleet size, flight rates, and program duration. The model takes a corporate finance mentality as far
as economic modeling and requires certain inputs such as financial ratios and rates (debt-to-equity
American Institute of Aeronautics and Astronautics
10
ratio, discount rates, etc.) to define the financial structure of the hypothetical company. Revenues
are calculated using this information along with nonrecurring costs, recurring costs, and market
demand elasticity. Government incentives can be modeled in the form of direct cash contributions,
tax-breaks, and loan guarantees. Outputs include the Net Present Value of the company, breakeven
point, internal rate of return, and price sensitivity information. Year-by-year detailed output
information is presented in traditional income, balance sheet, and cash flow statements.
M. ProbWorks
Tool Name: ProbWorks for ModelCenter®
Function: Probabilistic Simulation
Platform: Java Plug-In for ModelCenter®
Origination: Developed and commercially sold by PiBlue Software
Description: A suite of uncertainty and sensitivity analysis tools including Advanced Monte Carlo, Discrete
Probability Optimal Matching Distribution (DPOMD), Pareto Sensitivity, and Response Surface
Equation (RSE) Generator. The probabilistic capabilities in ProbWorks are useful when trying to
minimize the computational expense for Monte Carlo simulations, ranking the influence of input
variables, and design space approximation through Response Surface Methodologies (RSM). As a
ModelCenter® plug-in component, ProbWorks can be easily set up to perform these actions for
the integrated tools of I-RaCM. The inputs and outputs selected for a ProbWorks simulation can
be any combination of inputs and outputs from any tool in I-RaCM, giving the user unlimited
flexibility to investigate uncertainty and sensitivity of the model.
N. OptWorks Tool Name: OptWorks for ModelCenter®
Function: Optimization
Platform: Java Plug-In for ModelCenter®
Origination: Developed and commercially sold by PiBlue Software
Description: OptWorks is a suite of non-gradient-based optimization tools for use within Phoenix Integration's
ModelCenter® collaborative design framework. Each driver component has a particular benefit or
utility to different classes of problems. The individual drivers in the current release include
Genetic Algorithm (GA), Simulated Annealing (SA), Coordinate Pattern Search (with or without a
Compass Search option), Grid Search, Random Walk, and Random Search. The optimization
algorithms in the OptWorks suite are useful for a wide variety of complex optimization problems
in which the design space may be non-smooth or discontinuous. Within I-RaCM, OptWorks
allows for optimization of the global model. For example, the total program cost can be optimized
subject to schedule constraints by exploring the values of multiple technical inputs to the cost
models.
III. Lunar Exploration Architecture Example Case Studies
An implementation of I-RaCM consisting of a selection of the available tools was created to demonstrate the
basic capabilities and usefulness of the integrated model. Four case studies were conducted. The departure point for
all case studies was a baseline lunar exploration architecture in which no new technologies are required and a
moderate flight schedule is assumed. The first case study demonstrates the schedule optimization capability of
Stack’em whereby the schedule is automatically adjusted to meet a given budget. In the second case study, an
alternate architecture scenario in which four new technologies are developed, is compared with the baseline
architecture. The third case study compares a campaign with a more aggressive flight schedule to the baseline.
Lastly, a Monte Carlo simulation on the proxy performance outputs is conducted for the final case study yielding
probability distributions on the cost of the baseline architecture.
A. Baseline Architecture Overview
The modeled architecture draws heavily on the Exploration Systems Architecture Study (ESAS) released by
NASA in 2005 with the addition of a habitat, an unpressurized rover, and other surface system elements. The
purpose of the ESAS activity was to explore options for NASA’s initiative to return to the Moon.14
I-RaCM
modeling focused on the cost and reliability aspects of the in-space transportation and lunar surface elements
apportioned to lunar exploration. The intention was to isolate the lunar campaign from other possible associated
American Institute of Aeronautics and Astronautics
11
activities such as International Space Station support. The ESAS architecture is a “1.5 launch” approach which
incorporates Earth Orbit Rendezvous (EOR) en-route to the moon and Lunar Orbit Rendezvous (LOR) on the return
leg. All propulsive stages in this architecture use conventional chemical liquid rocket engines.
Earth to orbit launch for crew and cargo is accomplished using Space Shuttle-derived systems. The four
astronauts ride to orbit aboard the Crew Exploration Vehicle (CEV), a combination of an Apollo-like Crew Module
(CM) capsule and a Service Module (SM). The CEV is launched atop the Crew Launch Vehicle (CLV) which
consists of a Space Shuttle Solid Rocket Booster (SRB), and a newly-designed upperstage. Meanwhile, a second
launch vehicle, referred to as the Cargo Launch Vehicle (CaLV), boosts the Earth Departure Stage (EDS) and Lunar
Surface Access Module (LSAM) into LEO. Once in LEO, the EDS, LSAM, and CEV rendezvous and the EDS
performs a trans-lunar injection (TLI) burn to place the entire stack on a transfer orbit to the moon. The empty EDS
is then jettisoned.
When the LSAM and CEV arrive in the vicinity of the moon, the LSAM performs a lunar orbit insertion (LOI)
burn. The LSAM separates from the CEV in Low Lunar Orbit (LLO) and descends to the surface of the moon with
all four crew members aboard. Upon the conclusion of the surface stay, the ascent stage of the LSAM lifts off and
returns to LLO to rendezvous with the waiting CEV. The LSAM ascent stage is discarded, and the CEV SM
performs a Trans-Earth Injection (TEI) burn to place the vehicle on a return path to Earth.
As the CEV approaches Earth, the SM is jettisoned and the CEV CM enters the atmosphere directly. The blunt-
body capsule decelerates aerodynamically in the upper atmosphere, deploys parachutes once an appropriate Mach
number is reached, and eventually executes a land or water landing.
In addition to the primary transportation elements of this architecture described above, the case studies included
three additional elements. These were a surface habitat, an unpressurized rover, and a catch-all category for lunar
surface systems including extravehicular activity (EVA) elements ranging from pressure suits to tools, surface
power units, and lunar communications surface units. The LSAM is assumed to be designed to carry a substantial
payload to the lunar surface in addition to the human crew. This capability will enable delivery of items such as
rovers and habitats. Figure 2 summarizes the baseline architecture elements and some of their key characteristics.
The baseline lunar exploration program begins in 2007 and runs for 21 years until 2027. Optional technology
development, turned on and off through the TCE component, begins in 2007 as well. The first human lunar mission
is flown in 2018 and the full campaign runs until program end in 2027. The first mission is a lunar fly-by only and
does not include a lunar landing. After the fly-by, there is one lunar landing mission a year for the next three years.
During this time, two habitat modules, an unpressurized rover, and some surface systems equipment is delivered.
Two missions a year are conducted from that point forward until program end. Two additional habitats and three
additional rovers are landed by 2025, along with additional surface systems equipment needed for power and
communications. Sixteen total missions are conducted in the ten year campaign period.
American Institute of Aeronautics and Astronautics
12
Figure 2. Summary of elements of case study lunar exploration architecture
Includes
new in-space pressure
suit, EVA suit, and
various EVA tools and mobility aids.
Enables crew
to safely traverse approximately
10-15 km from the landing
site.
Designed to
support 4 crew members for
30 days on the lunar
surface.
Designed to
transport 4 crew members to
and from the lunar surface
from low lunar orbit (LLO).
Uses
LOX/CH4 propellants. Performs
propulsion, thermal
control, and power functions for the CEV.
Designed for
4 crew on lunar mission.
Reusable blunt-body
capsule with ablative TPS.
Uses
LOX/LH2 propellants.
Shuttle-
derived vehicle using 2 SRBs and
a LOX/LH2 core stage.
Shuttle-
derived vehicle using solid rocket
booster (SRB) plus
LOX/LH2 upperstage.
Other
201520142015201620152016201420152014IOC
201020102008200820132013200720072007DD Start Year
-7,100 lbs9,457 lbs8,812 lbs17,712 lbs42,450 lbs637,465 lbs226,650 lbsDry Mass
1
Lunar Surface Access Module (LSAM)
1
Surface Habitat
2
UnpressurizedRover
2
Lunar Crew Exploration
Vehicle (CEV) Crew Module (CM)
2
Lunar Crew Exploration
Vehicle (CEV)
Service Module (SM)
2---Pre-Phase A Duration
EVA and Surface Systems
Earth Departure
Stage (EDS)
Cargo Launch Vehicle (CaLV)
Crew Launch Vehicle (CLV)
Element Name
Includes
new in-space pressure
suit, EVA suit, and
various EVA tools and mobility aids.
Enables crew
to safely traverse approximately
10-15 km from the landing
site.
Designed to
support 4 crew members for
30 days on the lunar
surface.
Designed to
transport 4 crew members to
and from the lunar surface
from low lunar orbit (LLO).
Uses
LOX/CH4 propellants. Performs
propulsion, thermal
control, and power functions for the CEV.
Designed for
4 crew on lunar mission.
Reusable blunt-body
capsule with ablative TPS.
Uses
LOX/LH2 propellants.
Shuttle-
derived vehicle using 2 SRBs and
a LOX/LH2 core stage.
Shuttle-
derived vehicle using solid rocket
booster (SRB) plus
LOX/LH2 upperstage.
Other
201520142015201620152016201420152014IOC
201020102008200820132013200720072007DD Start Year
-7,100 lbs9,457 lbs8,812 lbs17,712 lbs42,450 lbs637,465 lbs226,650 lbsDry Mass
1
Lunar Surface Access Module (LSAM)
1
Surface Habitat
2
UnpressurizedRover
2
Lunar Crew Exploration
Vehicle (CEV) Crew Module (CM)
2
Lunar Crew Exploration
Vehicle (CEV)
Service Module (SM)
2---Pre-Phase A Duration
EVA and Surface Systems
Earth Departure
Stage (EDS)
Cargo Launch Vehicle (CaLV)
Crew Launch Vehicle (CLV)
Element Name
American Institute of Aeronautics and Astronautics
13
B. I-RaCM Tools for Case Studies
The selection, integration, and linking of I-RaCM tools depends on the specific architecture being modeled and
goals of the analysis. The pre-existing wrappers and ModelCenter® Link Editor allow the tools to be quickly
integrated. A subset of all available I-RaCM tools was selected to model the lunar exploration architecture in order
to demonstrate general functionality without overly complicating the discussion. The organization, relationship, and
flow between tools would likely be similar for other architectures. The tools integrated in this example
implementation of I-RaCM are the Campaign Manager, NAFCOM, FGOA, Reliability_Calc, TCE, TIM, Stack’em,
and ProbWorks. In addition, it was necessary to include one additional component to serve as a proxy for the
performance loop. For this example implementation, it was decided not to combine I-RaCM with traditional
conceptual design sizing and performance components (e.g. propulsion, mass estimating, aerodynamics, etc.).
However, a cost estimating process requires the outputs of the performance analysis as inputs, so it was necessary to
create a proxy for the performance loop. The Performance and Sizing Proxy is simply a spreadsheet of
representative performance analysis outputs to serve as inputs to the downstream cost and reliability estimating
tools. Values defining technical specifications for elements of the lunar exploration architecture such as mass,
power, volume, efficiency, and duration are entered into the spreadsheet. While the variable values do not change as
a result of an integrated performance loop, changes to performance outputs can be imitated by changing their values
within the spreadsheet.
Figure 3 is a screenshot of the complete I-RaCM in ModelCenter®. The topmost component is the Performance
and Sizing Proxy. The outputs from this component feed into NAFCOM cost models for elements of the lunar
architecture. The TCE tool informs the Technology Impact Matrix which technologies are active, and the
Technology Impact Matrix in turn adjusts the values of inputs to the cost and reliability tools. Reliability Calc passes
its calculated reliability outputs to Stack’em. The Campaign Manager, at right, provides mission model information
to Stack’em. Stack’em also collects cost outputs from the NAFCOM components, the FGOA components, and TCE.
Surrounding the entire I-RaCM integrated tool suite is a ProbWorks© Monte Carlo driver which is linked to the
inputs of the Performance and Sizing Proxy and the outputs of Stack’em in order to conduct probabilistic studies on
the performance inputs.
Figure 3: I-RaCM in ModelCenter®
American Institute of Aeronautics and Astronautics
14
The costs of three of the nine architecture elements are modeled explicitly in the I-RaCM prototype. Integrated
NAFCOM components are in this implementation of I-RaCM for the EDS, LSAM, and Habitat. Representative
performance loop outputs from the Performance and Sizing Proxy feed the appropriate NAFCOM component for
these elements. The NAFCOM outputs are then in turn passed to Stack’em. Elements not explicitly modeled with
NAFCOM components have fixed development and production cost values inputted directly into Stack’em. There is
no technology development in the baseline architecture and thus no technology cost.
The inputs to the I-RaCM model of the selected architecture were set to isolate the lunar campaign costs. Thus
the production cost of all elements is taken into account, but it is assumed that there is no development cost for the
launch vehicles. This is reflective of the real world situation where the CaLV and CLV are being developed in part
to service the International Space Station (ISS) and provide Earth-to-orbit heavy lift capability. Development of the
CEV Command Module is also assumed to be offset by development of that spacecraft for crew delivery to the ISS.
Thus the cost inputs to I-RaCM are only those additional costs needed to modify the ISS version of the CEV CM to
the lunar CEV CM. Similarly, facilities and ground equipment cost for the CaLV and CLV is considered to be only
any modification cost to the existing facilities in order to launch the lunar missions. There are two instances of
FGOA in the lunar exploration architecture I-RaCM prototype, one for the CaLV and one for the CLV. The inputs
to each of these FGOA models have been set to account for only the modification costs.
Reliability of a lunar mission depends on all elements functioning properly, so all elements are included in
reliability determination. The Reliability Calc tool has fault tree and event sequence diagram logic for the CaLV,
EDS, CLV, LSAM, Habitat, CEV SM, and CEV CM. Reliability of the unpressurized rover and other surface
systems is not calculated within the Reliability Calc component, but a placeholder with a fixed value is included
within Stack’em.
C. Baseline Architecture Results
While all cost inputs and other data analysis data was solely derived from publicly available sources, the cost
results published herein have been normalized to avoid insinuation that any sensitive information is presented.
Figure 4, a “sand chart” generated by Stack’em, depicts a breakdown of the baseline architecture costs by element
over the duration of the campaign. The red line on the chart depicts the program budget, a year by year dollar
amount entered into Stack’em. With the notional budget shown, it is clear that the program violates the allocated
budget slightly in 2007 through 2009 and more dramatically from 2014 through 2016.
A Gantt chart is available within Stack’em to help the user visualize the schedule for each program element. The
Gantt chart for the baseline architecture and campaign is given in Figure 5. The schedule shows the duration of three
major phases of the product lifecycle; design and development, test and evaluation, and production.
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
2007
2009
2011
2013
2015
2017
2019
2021
2023
2025
2027
No
rma
lize
d Y
ea
rly
Co
st
an
d B
ud
ge
t Program Wrap
Reserves
Facilities & Operations
Technology Maturation
Other Surf. Sys.
Unpress Rover
Hab
LSAM
CEV SM
CEV CM
EDS
CaLV
CLV
Budget
Figure 4: “Sand chart” of total program costs for baseline architecture and campaign
American Institute of Aeronautics and Astronautics
15
Stack’em also outputs a chart of total cost broken down by category. Figure 6 shows this chart for the baseline
lunar exploration architecture. As can be seen, facilities and GSE cost makes up only a small portion of total cost in
the early years of the program. This is due to the relatively small cost of modifying the assumed existing CLV and
CaLV facilities to accommodate lunar missions. Design and Development (DD) cost is primarily apportioned
through the earlier years of the program and does not account for as much of the total cost as does production. DD
dollars after 2017 are due to sustaining engineering cost. Notice that production begins prior to the first mission in
2018. More information about the production schedule can be gleaned from the inventory chart.
The inventory chart within Stack’em shows the user how many units of each element have been produced at any
given time in the life of the program. Given their start date and production rates, elements may build up a small
queue over time. Fractional units can be produced in a given year, accounting for non-whole numbers in the
inventory chart. For the baseline scenario, as shown in Figure 7, several of the elements reach an inventory of four
or more units. Constraints may be set by the user within Stack’em, limiting the total number of units which may be
held in inventory, but such constraints were not set for this analysis.
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
2007
2009
2011
2013
2015
2017
2018
2020
2022
2024
2026
No
rma
lize
d Y
ea
rly
Co
st
by
Ca
teg
ory
Technology
Facilities
Production
T&E
DD
Figure 6: Costs by category for baseline architecture and campaign
2007 2009 2011 2013 2015 2017 2019 2021 2023 2025 2027
CLV
CaLV
EDS
CEV CM
CEV SM
LSAM
Hab
Unpress Rover
Other Surf.
Design and Dev Test and Eval Production
Figure 5: Gantt chart for baseline architecture and campaign
American Institute of Aeronautics and Astronautics
16
Within Stack’em the user can optionally run, via a macro button, a built-in Monte Carlo driver (ProbWorks by
PiBlue Software) to conduct a probabilistic analysis on the element costs. The Monte Carlo simulation samples from
distributions on the input costs of the elements to probabilistically sum the total program cost. The results of this
simulation provide the user with a distribution of total program cost where the variability is due to uncertainty in the
cost inputs. A graphical chart also displays the distributions of cost for each element over the duration of the
campaign, providing the user with information about which elements contribute the most to the uncertainty in total
program cost.
For the current version of Stack’em, the Monte Carlo simulation can be set up automatically using a macro, but
the format of the output chart must be set up mostly manually. The inputs to Stack’em defining the cost distributions
may be triangular, normal, uniform, exponential, Weibull, lognormal, or beta. These cost distribution parameters
may be passed directly from the cost estimating tools (when they are run probabilistically) or input by the user.
For the baseline lunar exploration architecture, triangular distributions were placed on the input costs. The
distributions used were notional for the purpose of illustration. Figure 8 shows the resulting cost distributions. The
minimum and maximum of the output distribution for each element is represented by the bounds of the bar. Points
near the mean of the distribution appear as a darker blue, while the tails of the distribution are lighter in color. The
orange horizontal lines in the bars represent a particular user-specified percentile of the distribution, in this case the
70th
percentile. At a glance the user can see that the LSAM is the most costly element and also one of the most
uncertain. The CLV and CaLV costs are also more uncertain than the remaining elements. The location of the 70th
percentile and mean for the LSAM and CaLV distributions indicate that these distributions are left skewed. As such,
there is a greater probability that their costs could be toward the low side, but also a higher limit to the potential cost
of that element.
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
CLV
CaL
VED
S
CEV
CM
CEV
SM
LSAM
Hab
Unp
ress
Rov
er
Oth
er S
urf.
Sys.
No
rmalized
To
tal C
ost
Dis
trib
uti
on
s
an
d 7
0th
Perc
en
tile
Figure 8: Total cost distributions for Architecture elements
0
1
2
3
4
5
6
7
2012 2015 2018 2021 2024 2027
Nu
mb
er
of
Un
its in
Inven
tory
at
Year
En
d
CLV
CaLV
EDS
CEV CM
CEV SM
LSAM
Habitat
Unpress. Rover
Other Surf. Sys.
Figure 7: Inventory of produced elements over campaign lifetime
American Institute of Aeronautics and Astronautics
17
The mean total program cost from the Stack’em Monte Carlo analysis was three percent higher than the
deterministic point estimate. The 70th
percentile value for total program cost is also available from the Monte Carlo
simulation within Stack’em. These values provide a more robust estimate of cost to the analyst by accounting for
uncertainty in the input estimates.
Loss of mission (LOM) and loss of crew (LOC) calculations made by the Reliability Calc tool are compiled
within Stack’em. Reliability calculation of the baseline architecture considered Lunar Orbit Rendezvous (LOR) and
Earth Orbit Rendezvous (EOR) maneuver reliabilities in addition to the element reliabilities shown in Figure 9. The
LSAM/Hab and CEV are the greatest contributors to failure probability. Loss of crew probability is also calculated
on a per mission basis and then program lifetime reliabilities are calculated within Stack’em from the individual
LOM and LOC estimates.
D. Case Study 1: Program Expenditure Optimization
Recall that the cost of the baseline architecture did not fit under the notional budget curve. The total available
budget, however, well exceeds the total cost of the program. This suggests that the schedule may be adjusted to fit
the budget curve. The Coordinate Pattern Search optimizer of the PiBlue Software OptWorks© Excel® plug-in has
been set up within Stack’em to automatically fit the schedule to the available budget. Running the optimizer yielded
the “sand chart” of Figure 10 after about two minutes. The budget total is nearly the same (not exactly due to
sustaining engineering costs), but it now fits under the budget in all years. Comparing the optimized results to the
previous non-optimized results, the strategy for meeting the budget can be seen. It involves, among other changes,
delaying and condensing production of the CLV and CaLV, producing the surface systems earlier in time, and more
evenly distributing the Habitat development and production costs over time.
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
2007
2009
2011
2013
2015
2017
2019
2021
2023
2025
2027
No
rma
lize
d Y
earl
y C
os
t a
nd
Bu
dg
et Program Wrap
Reserves
Facilities & Operations
Technology Maturation
Other Surf. Sys.
Unpress Rover
Hab
LSAM
CEV SM
CEV CM
EDS
CaLV
CLV
Budget
Figure 10: Optimized “Sand chart” of total program costs for baseline architecture and campaign
0% 20% 40% 60% 80% 100%
LOM
CLV
CaLV & EDS
LSAM & Hab
CEV-SM & CM
Unpress Rover
EVA
Figure 9: Contribution of elements to a single LOM event for the baseline architecture
American Institute of Aeronautics and Astronautics
18
Optimization of the schedule will not always result in a program that fits under the budget curve. Insufficient
available budget or constraints on how much the schedule of each element can shift may prevent from meeting the
budget. Optimization will, however, almost always result in a program that better fits the budget as compared to the
baseline. The post-optimization results can then be evaluated to identify potential solutions to complete the program.
Examples of potential solutions include adjustments to the campaign, reduction in the technical requirements,
targeted increases in the budget, and element design changes to decrease individual element costs. The integrated
environment of I-RaCM allows for diverse types of potential solutions to be quickly evaluated.
E. Case Study 2: Technology Infusion
Four technologies were selected for the technology infusion case study; composite propellant tanks, high thrust
to weight engine, high density batteries, and integrated vehicle health monitoring (IVHM). These technologies were
intended to be somewhat generic with the intention of demonstrating I-RaCM rather than producing comprehensive
technology application results for a lunar exploration architecture. The technology impact matrix for these
technologies is shown in Table 2. When technologies are switched on within the TCE, the inputs passed to
NAFCOM components from the performance loop proxy and other NAFCOM variable values are adjusted by
means of the Technology Impact Matrix. The adjustment could be in the form of a multiplicative factor or a literal
variable setting. The technologies may also adjust the reliability calculations of Reliability_Calc. The cost of the
technology development itself computed by the TCE is passed to Stack’em.
Table 2: Technology Impact Matrix for Technology Infusion Case Study
Impacted Variable Technology Impact (Tech Off / Tech On)
Element Subsystem WBS Item or Variable
Composite
Propellant
Tanks
High
T/W
Engine
High
Density
Batteries
IVHM
Tank Structural Mass Multiplier -- / -6% Structures
Degree of New Design Setting 5 / 6
Engine Mass Multiplier -- / -25%
DD Complexity Value 0.3 / 0.4
EDS Liquid
Rocket
Engine Unit Complexity Value 1 / 1.25
EPD Mass Multiplier -- / -10%
Storage Capacity Multiplier -- / +5% Habitat Electrical
Power Manufacturing Methods Setting 2 / 1
EPD Mass Multiplier -- / -5%
Storage Capacity Multiplier -- / +5% Electrical
Power Manufacturing Methods Setting 3 / 2
LOM Engine Value .0026 /
.0015
LOM Power Systems Value .001 /
.0007 Reliability
LOM RCS Value .0004 /
.0002
CCDH Mass Multiplier -- / +10%
Degree of New Design Setting 6 / 8
LSAM
Ascent
Stage
CCDH
Eng. Management Setting 2 / 4
Tank Structural Mass Multiplier -- / -6% Propulsion
Degree of New Design Setting 6 / 7
EPD Mass Multiplier -- / -5%
Storage Capacity Multiplier -- / +5% Electrical
Power Manufacturing Methods Setting 3 / 2
LOM Engine Value .000022 /
.000015
LSAM
Descent
Stage
Reliability
LOM Power Systems Value .0000395
/ .00002
American Institute of Aeronautics and Astronautics
19
When technologies are added to an architecture, their development costs show up on the cost by category chart
within Stack’em. As Figure 11 shows, technology development costs make only a small contribution to the total cost
of the program for the case study. However, the cost to develop the technologies is only part of the complete
technology cost. The use of new technologies can have implications for the development and production of the
various elements as well, and these differences in element cost are modeled by the interconnected tools within I-
RaCM.
The cost differences due to technology infusion versus the baseline are given in Table 3. For the LSAM and EDS
there is a modest increase in cost with the new technologies due to mass and complexity differences. The Habitat is
nearly the same cost with and without the technologies applied. For the entire program, the addition of technologies
increases cost over the life of the program by 2.3 percent due to the cost of the technology development and the
increased cost of the elements themselves.
In addition to its cost implications, the benefits of the IVHM technology on reliability are captured by I-RaCM.
The IVHM system yields a modest improvement in reliability of the LSAM due to increases in reliability of the
Ascent Stage engine, power systems, and reaction control system; and the Descent stage engine and power systems.
Over the 16 flight lifetime of the campaign, the reliability improvements are more pronounced decreasing the
lifetime LOC events by just over 11 percent.
F. Case Study 3: Baseline Versus Aggressive Campaign
For this case study, a more aggressive mission campaign was entered into the Campaign Manager component
and I-RaCM was run to determine the cost comparison of this campaign to the baseline. The alternative, more
aggressive, campaign begins in 2018 and has the same duration (10 years) as the baseline. The alternative campaign
has a lunar landing scheduled in 2018 instead of just a lunar fly-by, plans two flights per years starting in 2020
rather than 2022, and goes to three flights per year for four years beginning in 2024. A total of 22 missions are on
the manifest compared to 16. The more aggressive campaign also lands five unpressurized rovers instead of four,
and lands additional surface system equipment.
Table 3: Cost Results of Technology Infusion Versus the Baseline Case
Campaign Element Total Cost Over Campaign
With Technology Development
EDS + 5.1 %
LSAM + 3.8 %
Habitat - 0.2 %
Program Total + 2.3 %
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
2007
2009
2011
2013
2015
2017
2018
2020
2022
2024
2026
No
rma
lize
d Y
ea
rly
Co
st
by
Ca
teg
ory
Technology
Facilities
Production
T&E
DD
Figure 11: Costs by category for architecture with technology development illustrating additional
technology development costs in early years
American Institute of Aeronautics and Astronautics
20
The total cost of the aggressive campaign is about twenty percent more than the baseline campaign. Indeed, as
Figure 12 shows, the aggressive campaign busts the available budget. Iterating with the campaign manager and
Stack’em schedule optimizer would allow the user to consider alternate campaigns that are more aggressive than the
baseline campaign that might still fit under the budget curve.
G. Case Study 4: Probabilistic Analysis of Performance Variables
A Monte Carlo simulation was performed on the variables input to the I-RaCM cost and reliability tools from the
performance disciplines, represented by the Performance Loop Outputs Proxy, for this last case study. Probabilistic
simulation in this manner has value in that it accounts for uncertainty in the physical design of the architecture
elements and provides an understanding of the resulting distribution of cost due to this uncertainty. The Monte Carlo
simulation is driven within I-RaCM by the PiBlue ProbWorks© Monte Carlo driver plug-in for ModelCenter®.
A histogram of total program cost for the baseline lunar exploration program is shown in Figure 13. The x-axis
on this chart would normally be presented in billions of dollars, but again has been normalized to completely avoid
any potential confusion about data sources. Much information of value can be gleaned from the shape and percentile
values of the distribution. This particular distribution is fairly tight and centered about the mean, indicating only a
small amount of uncertainty in the cost estimate.
0
5
10
15
20
25
30
35
40
0.992 0.994 0.996 0.997 0.999 1.001 1.003 1.004 1.006
Normalized Total Program Cost
Fre
qu
en
cy
Mean
Figure 13: Histogram of program total cost results from Monte Carlo simulation on performance variables
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
1.1
2007
2009
2011
2013
2015
2017
2019
2021
2023
2025
2027
Yea
rly C
os
t an
d B
ud
ge
t ($
M)
Program Wrap
Reserves
Facilities & Operations
Technology Maturation
Other Surf. Sys.
Unpress Rover
Hab
LSAM
CEV SM
CEV CM
EDS
CaLV
CLV
Budget
Figure 12: “Sand chart” of total program costs for alternative aggressive architecture and campaign
American Institute of Aeronautics and Astronautics
21
Histograms of element total cost over the length of the program for the EDS, LSAM, and Habitat are shown in
Figure 14. The mean cost of these individual elements is also very close to the deterministic values. This is likely
due to the triangular distributions placed on the performance parameters which were evenly distributed. In more
complex studies, these distributions may not be evenly distributed or the combinatorial effects of variable
interactions could result in non-centered output distributions.
Two Monte Carlo simulations have been discussed in the case studies section of this report and there is an
important distinction between the two worth reiterating. The Monte Carlo driver on the performance output
variables addresses the uncertainty in the performance parameters (mass, power, thrust, etc.), while the Monte Carlo
driver internal to Stack’em addresses the uncertainty in cost estimates. These two simulations could be combined to
address both types of uncertainty simultaneously. In such a scenario, the internal Stack’em Monte Carlo simulation
would be run once for every trial of the external Monte Carlo simulation on the performance parameters.
IV. Summary
The Integrated Risk and Cost Model meets the need for rapid and comprehensive cost and risk assessment early
in the design of a new system. I-RaCM integrates a diverse set of new and industry-standard tools for analysis of life
cycle cost, operations, reliability modeling, and estimation of technology development costs. I-RaCM contains
several novel tools and capabilities that enhance cost/risk insight and facilitate rapid trade space investigation. The
Remix Wrapper Generator provides a previously unavailable capability to integrate the NAFCOM and SEER-H cost
tools in the conceptual design process via ModelCenter®. The Technology Cost Estimator prototype demonstrates
the technical feasibility and usefulness of an early technology development cost estimation tool. Stack’em provides
a means for analysts to view key outputs and provides functionality for optimization of program expenditures and
probabilistic evaluation of program cost estimates.
Case studies conducted of a modern-day lunar exploration architecture demonstrated the functionality of I-
RaCM and the novel capabilities of the integrated tools. The case studies demonstrated basic cost and reliability
analysis, program expenditure optimization, technology infusion, campaign excursions, and analysis of performance
0
10
20
30
40
50
60
70
80
0.962 0.978 0.995 1.012 1.028 1.045
EDS Normalized Total Cost
Fre
qu
en
cy
Mean
0
10
20
30
40
50
60
0.982 0.990 0.998 1.005 1.013 1.021
LSAM Normalized Total Cost
Fre
qu
en
cy
Mean
0
10
20
30
40
50
60
70
0.979 0.988 0.996 1.004 1.013 1.021
Habitat Normalized Total Cost
Fre
qu
en
cy
Mean
Figure 14: Histograms of total cost over the duration of the baseline lunar exploration program for the EDS,
LSAM, and Habitat elements. Results from Monte Carlo simulation on performance variables
American Institute of Aeronautics and Astronautics
22
variable uncertainty on cost outputs. I-RaCM is a continuing development, evolving to include more analysis tools
and visualization capability, with plans in place for more advanced reliability analysis, availability and performance
tools, and discrete event simulation of operations processing. With these upgrades, I-RaCM will continue to improve
conceptual design outcomes.
V. Acknowledgements
The authors would like to thank NASA personnel and others who have contributed to the development of I-
RaCM. Initial conception and development of I-RaCM was conducted by SpaceWorks Engineering in fulfillment of
contract requirements for a 2006 Phase I NASA SBIR. Sponsorship and financial support was provided through this
SBIR award, contract number NNM07AA49C. The authors are grateful to the NASA Contract Officer’s Technical
Representative, Mr. Dan O’Neil at NASA Marshall Space Flight Center, whose shared experience with multi-
disciplinary modeling continues to guide development and use of I-RaCM. The authors would also like to
acknowledge Mr. J.D. Reeves of NASA Langley Research Center for review of a prototype version of I-RaCM and
useful feedback. Technical and organizational project support was provided by Mr. Jon Wallace and Dr. John Olds
of SpaceWorks Engineering, Inc. The authors are thankful for the contributions of these two individuals and the
entire SpaceWorks Engineering staff.
References 1Bradford, J. E., Eklund, D. R., Charania, A., Wallace, J. G., "Quicksat: A Two-Stage to Orbit Reusable Launch Vehicle
Utilizing Air-Breathing Propulsion for Responsive Space Access," AIAA-2004-5950, Space 2004 Conference and Exhibit, San
Diego, California, September 28-30, 2004. 2Bradford, J. E., Eklund, D., "Automated Hypersonic Launch Vehicle Design Using ModelCenter," GT-SSEC-B.1, First
Annual Georgia Institute of Technology Space Systems Engineering Conference, Atlanta, Georgia, November 8-10, 2005. 3Bradford, J. E., Olds, J. R., Wallace, J. G., "A Two-Stage Spacelift Concept Using Air-Breathing Propulsion and Falcon-
Derived Hardware," 54th Joint Army-Navy-NASA-Air Force (JANNAF) Propulsion Meeting (JPM) and 5th Modeling and
Simulation / 3rd Liquid Propulsion / 2nd Spacecraft Propulsion Joint Subcommittee Meeting, Denver, Colorado, May 14-18,
2007. 4St. Germain, B., Olds, J. R., Bradford, J. E., Charania, A., DePasquale, D., Schaffer, M., and Wallace, J. G., "Utilizing Lunar
Architecture Transportation Elements for Mars Exploration," AIAA-2007-6136, Space 2007, Long Beach, California, September
18 - 20, 2007. 5Charania, A., Crocker, A., Olds, J., Bradford, J., "A Method For Strategic Technology Investment Prioritization For
Advanced Space Transportation Systems," IAF-01-U.2.06, 52nd International Astronautical Congress, Toulouse, France, October
1-5, 2001. 6Charania, A., Olds, J.,"Application of the Abbreviated Technology Identification, Evaluation, and Selection (ATIES)
Methodology to a Mars Orbit Basing (MOB) Solar Clipper Architecture," IAC-02-U.5.01, 53rd International Astronautical
Congress, The World Space Congress - 2002, Houston, Texas, October 10-19, 2002. 7Crocker, A., Charania, A., Olds, J., "An Introduction to the ROSETTA Modeling Process for Advanced Space
Transportation Technology Investment," AIAA-2001-4625, Space 2001 Conference and Exposition, Albuquerque, NM, August
28-30, 2001. 8St. Germain, B., Charania, A., Olds, J. R., "A Stochastic Process for Prioritizing Lunar Exploration Technologies," AIAA-
2005-6607, Space 2005, Long Beach, California, August 30 - September 1, 2005. 9DePasquale, D., Kelly, M., Charania, A., Olds, J. R. "Activity-Based Simulation of Future Launch Vehicle Ground
Operations," AIAA-2008-7660, Space 2008, San Diego, California, September 9 - September 11, 2008. 10Charania, A., DePasquale, J., "Simulating the Dynamic Marketplace: An Introduction to the Nodal Economic Space
Commerce (NESC) Model," AIAA-2005-6617, Space 2005, Long Beach, California, August 30 - September 1, 2005. 11DePasquale, D., Charania, A., "Winning in the Next Space Market: Prospects for Financial Success of Commercial
Transportation Services to the International Space Station (ISS)," SEI Case Study, January 2007. 12DePasquale, D., Charania, A., Olds, J. R., "Sub-Orbital Space Tourism: Predictions of the Future Marketplace Using Agent-
Based Modeling," IAC-06-E3.4, 57th International Astronautical Congress, Valencia, Spain, October 2-6, 2006. 13Charania, A., Olds, J. R., "Optimization of a Future RLV Business Case Using Multiple Strategic Market Prices," IAC-02-
IAA.1.1.07, 53rd International Astronautical Congress, The World Space Congress - 2002, Houston, Texas, October 10-19, 2002. 14National Aeronautics and Space Administration, “NASA’s Exploration Systems Architecture Study,” Final Report, NASA-
TM-2005-214062, November 2005.