stracener_emis 7305/5305_spr08_04.17.08 1 system availability modeling dr. jerrell t. stracener, sae...

1

Stracener_EMIS 7305/5305_Spr08_04.17.08

System Availability Modeling

Dr. Jerrell T. Stracener, SAE Fellow

Leadership in Engineering

EMIS 7305/5305Systems Reliability, Supportability and Availability Analysis

Systems Engineering ProgramDepartment of Engineering Management, Information and Systems

2


Availability Modeling

• Requirements and Figures of Merit• Analytical versus Simulation Modeling• Availability Model Development• Blue Flame Aircraft Case Study• Summary and Discussion

3


Requirements and Figures of Merit

Why Model?

4


Availability Analysis

provides a mathematical basis for evaluating system design and development decisions based on system level performance measures in order to influence the air vehicle design concurrently with support system design.

5


System Design Evaluation Categories

Operational Effectiveness Evaluation

“To what degree does this system satisfactorily support mission accomplishment when used by representative personnel in the expected or potential environment for operational employment of the system considering organization, doctrine, tactics, survivability, vulnerability, and threat?”

Operational Suitability Evaluation

“To what degree can this system satisfactorily be deployed considering availability, compatibility, transportability, interoperability, reliability, wartime usage rates, maintainability, safety, human factors, manpower supportability, documentation and training requirements?”

Functional Effectiveness Evaluation

“How and to what degree will this system satisfactorily contribute to the required mission(s) in the predicted operational environment?”

(a combined, system-level assessment)

ScheduleEvaluation

CostEvaluation

•Requirements•MOEs/MOSs•Critical Issues•Test Objectives•Thresholds•Deficiency andFailure Tracking

System/Segment (Type A)Functional Baseline

6


RM&S Integration into System Engineering ProcessS

yste

m E

ng

ineeri

ng

, LS

A,

Inte

gra

ted

Log

isti

cs S

up

port

TechnicalDisciplines

• Operations Analysis• Life Cycle Cost• Survivability/

Vulnerability• Safety• Reliability/Parts

Standardization• Maintainability• Human Factors• Maintenance

Concept/Plan• Spares Provisioning• Support Equipment• Training equipment• Training• Technical Publications• Packaging, Handling,

Storage & Transportation

• Facilities• Manpower

Requirements & Personnel

• Logistics Support Resource Funding

• Energy Management• Computer Resources• ILS Test & Evaluation• ILS Planning• ILS Management

•Logistics Concept Planning and Development

•Life Cycle Cost Goals•Supportability Specifications

Pre-FSD

•Update ILS Plans•Quantification of Support

Requirements• Integration of Support Studies

and Analyses•Design Support Trade Off

Studies•Evaluation of Support System

Effectiveness

Transition toProduction

• Identification & Resolution of Support Problems

•Analyses for Operational/Support Concept ChangesEvaluation of System Mods Impacts on Support

Operations

FSD

System

SupportSystem

TrainingSystem

7


The System View

Product

•Spares•Technical Publications•Training•Support Equipment

•Availability•Sortie Generation Rates•Basing

•Reliability•Maintainability•Supportability•Testability

•Organization•Requirements•Schedule Maintenance•Unscheduled Maintenance

Operational

Concept

MaintenanceConcept

SupportConcept

8


Analytical versus Simulation

9


General Modeling Options

• Analytical Representations– Mathematical formulas and symbolic models– May use computers to process the formulas

• Computer Simulations– Imitation of the physical phenomena(movement,

war, performance overtime) using computer generated activities and results

– human decision making represented by pre-programmed and/or probabilistic decision rules

• Assemblage of Gaming People and Tools– Human-based “game playing” to achieve insights

(e.g. war games)

• Field Experiments– Replications of a physical situation under controlled

and limited scale environments to estimate total system level performance

10


When simulation models make sense

• When mathematical models do not exist, or analytical methods of solving them have not yet been developed

• When analytical methods are available, but mathematical solution methods are too complex to use

• When analytical solutions exist and are possible, but are beyond the mathematical capabilities of available personnel

11


When simulation models make sense

• When it is desired to observed a simulated history of the process over a period of time in addition to estimating relevant parameters

• When it may be the only possibility because of difficulty in conducting experiments and observing phenomena in their actual environment

• When time compression may be required for systems over long time frames

12


Advantages of Simulation

• Permits controlled experimentation with:– consideration of many factors– manipulation of many individual units– ability to consider alternative polices– little or no disturbance of the actual system

• Effective training tool• Provides operational insight• May dispel operational myths

13


Advantages of Simulation

• May make middle management more effective

• May be the only way to solve problem

14


Disadvantages of Simulation

• Costly (very costly?)• Uses scarce and expensive resources• Requires fast, high capacity computers

(use of PC’s?)• Takes a long time to develop• May hide critical assumptions• May require expensive field studies• Very much dependent on availability of

data and is validity

15


Typical A/R/S Analysis Models

• Analytical Models– Inherent Availability Models– Expected Value Models– Stochastic/Markov Models

SAVE (System Availability Estimator)– Differential Equation Models– Parametric Models

• Simulation Models (Mainframe & PC-based)– Top-Level A/R/S Models

Theater Simulation of Air Base Resources (TSAR)-Rand Corp.

Douglas Aircraft Company Availability Model (DACAM) System Inventory Analysis Model (SIAM)

– More detailed A/R/S ModelsModified Logistics Composite Model (LCOM)-USAFComprehensive A/C Support Effectivenes Eval.

(CASEE)Model -USNavy

16


When Simulation Models Make Sense(An Analyst’s Checklist)

• When mathematical models do not exist, or analytical methods of solving them have not yet been developed

• When analytical methods are available, but mathematical solution methods are too complex to use

• When analytical solutions exist and are possible, but are beyond the mathematical capabilities of available personnel

• When it is desired to observe a simulated history of the process over a period of time in addition to estimating relevant parameters

• When it may be the only possibility because of difficulty in conducting experiments and observing phenomena in their actual environment

• When time compression may be required for systems or processes over long time frames

17


System Life Cycle Utility of Models/AnalysesConcept Explore.

DEM/VAL FSED PRODUCT DEPLOY

Statistical Analysis Capability Low Medium High High HighReliability Growth Models Low Medium High - -

LCC/CERs/O&S/DTC Models Low* Medium* High Medium LowFMECA Process Low* Medium* High Low** Low**

Basic Rel & Maint Anal. Tools Medium* Medium* High Low -Airbase Operations Model Low* Low* High Low** Low**LCOM/Airbase Ops Model Low* Low* High Low** Low**

Integ. CALS Status Reporting Low* Low* High High LowMsn Effect. & Supportability Medium* Medium* High Low -

Probability of Mission Success Medium* Medium* High Low -Analyt. Avail. (Markov) Models Medium* Medium* High - -Paramet. R&M Prediction Mod Medium* High* High - -

Enhanced ILS/LSAR System Low* Medium* High Medium Low**Reliability Centered Mainten. - Low* High Medium Low**

Enhanced RCM Process - Low* High Medium Low**High Level A/R/S Models High* High* Medium Low** Low**

LCOM w/Model Mgmt. System Low* Low* High High** High**Comprehensive Autom. Supp. Low* Medium* High Medium Low**

RMS Design Training Pkgs. Low Medium High Medium LowCALS/CASE (Full Up Capab.) Low* Medium* High High Medium*

* - High level analysis ** - ECP/Changes/Problem Resolution

18


Thoughts To Remember

• The overall objective of availability modeling and analysis is to provide support to the system design, development, and deployment process in order to influence system design by considering all aspects of its reliability, maintainability, and support system characteristics

• The objective remains unaffected by the choice of using one model solution technique (e.g. simulation) over the other.

• The efficacy of choosing one method over the other will be influenced primarily by outside factors (e.g. cost, schedule, availability of data, personnel and facility capabilities).

19


Availability Model Development

20


Model Development Overview

• Analysis Objectives• Analysis Planning• Development Approach• Development Considerations• Inputs and Outputs• Data Requirements• Algorithm Development• Implementation Examples

21


RMS Analysis Objectives

• Specification Requirements Evaluation– Requirement Integration - Conflicts? Attainable?– Verify and Demonstrate Compliance– Verify Demonstrate Adequacy of Logistics Support

• Support System Design Influence– Evaluate Impacts of Changes to Operation and Maintenance Concepts– Analyze & Evaluate Operational Suitability– Support Functional Trade-off Analyses on Alternative Designs

• System Design Assessment– Examine the Total Picture at the System Level– Address Impacts of All Variables at once– Evaluate Impacts of Flight/Scenario/Usage Rate Changes

• Management Visibility– Provide Useful Predictions for All Levels of Management– Assist Management in Identification and Resolution of Reliability,

Maintainability, and Supportability Issues

22


RMS Analysis Objectives by Program Phase

• Concept Definition– Support Contractual Requirements Analysis

• Examine Operations, Maintenance, and Support Concepts– Support Design Concept Trade-off Studies– Identify Cost, Schedule, Risk, and Support Drivers

• Demonstration/Validation– Refine Concept Definitions– Support Requirements Allocation Process– Provide Capability to Influence Design– Estimate Fielded System performance Levels

• Full-Scale Engineering Development– Support Detailed Trade-off Studies– Establish Support System Requirements Baseline– Assess/Validate Operations, Maintenance, Support Concepts

• Production and Deployment– Asses Fielded System Performance Levels– Refine Support Concepts/Levels– Identify System Improvement Requirements

23


RMS Analysis Planning Considerations

• Critical Issues

•Objectives

•MOEs & MOSs

•Success Criteria

•Schedule

• Test Design

•Analysis Plan

•Data Collection & Management Plan

•Test Execution Plan

•Documentation Plan

•Test and Evaluation Master Plan

•Where does data come from?

–Experiment?

–Field tests?

–Previous experience?

–Simulation?

–Other resources?

•What will data be used for?

•How will data be collected and managed?

•What tests/simulations need to be executed, and when?

•How will results be dev. and rec?

•How does everything fit together to meet the system test & eval. objectives?

Evaluate A/R/S Analysis Reqs.

Develop Test /Analysis Plans

24


RMS Model Development Approach

1. Define Model Elements and Specificationsi. Operational Activity Elemanet Specificationsii. System State Conditions and Attribute Specificationsiii. Operational Activity Demand Generationiv. System Component Level of Detail Determinationv. Support System Resource Definition and Specifications

2. Define Model Structurei. Model Processing Definition(s)ii. System Failure Processingiii. System Unscheduled Maintenance Processingiv. Model Inputsv. Model Outputs

3. Implement Model Structure on the Computeri. Model Activitiesii. Model Output Measure Calculation Implementation

4. Perform Full Model Test & Eval. Using Sample Data5. Install Model at User Site and Perform Checkout,

Train Users

25


Probabilistic Modeling(probabilistic analysis)

• Purpose: To simulate probabilistic situations using a random number generator and the cumulative probability distribution of interest.

• Example: Distribution of unscheduled maintenance times:no action required (none), repair in place (RIP), remove and replace (R&R), and cannot duplicate (CND)

Maintanance Type

Occurrence Probability

Cumulative Probability

None 0.9 0.9RIP 0.05 0.95R&R 0.03 0.98CND 0.02 1

Time Period

Random Number

Type of Maintenance

1 0.09 None2 0.93 RIP3 0.42 None4 0.97 R&R5 0.99 CND6 0.27 None

26


Analysis/Model Development Considerations

• Data Input/Output Formats• Data and Output Result Configuration Management &

Control• Input/Output Data Approval by Management• Baseline and Excursion Data Definitions/Conditions• Data Screening/Editing Capabilities• Model Restart Capabilities• Ease of Development and Modification• Transparency to the Users (changes to system and

data)• Degree of integration with other models and Analyses• Convenient Man-in-the-Loop Interfaces• Growth/Flexibility/Change Capabilities• Others

27


Typical RMS Model Requirements

• Work Unit Code (WUC) Structure– Total system WUC structure– Two Digit level definitions (or to levels of interest)

• Probability Distributions fro Activity Times (by WUC)– Mission durations and types– Trouble-shooting times– On/off aircraft repair times– Remove, replace, checkout times– Delay times (spares, personnel, equipment)– Service and turnaround times– Preflight and return service times

• Probabilities (by WUC)– Probability of in-flight failure (gripe)– Spares, personnel, equipment availability – when called– On equipment vs. off-equipment repair rates– No defect found rates

28


Input Data Sources & Parameters

• AFR 66-1 (Maintenance Data Collection System- MDCS) Data Elements

• CORE Automated Maintenance System (CAMS)

• AFR 65-110 (air Vehicle Inventory Status and Reporting System (AVISURS)

• Others

• “Reliability” in terms of MTBM– Types 1,2, & 6

• “On-equipment” & “Off-equipment” Maintenance Action Definitions:– Repair in Place– Cannot Duplicate– Bench Check –- Repair– Bench Check –- Serviceable– Not Repairable in this Station

• Work Unit Code Definitions• Others

Air Force RAM Data Sources Model Data Element Definitions derived from Air Force Terms

29


Typical RMS Output Parameters(for sensitivity analysis)

• Availability Parameters– Average mission capable rates (full, partial, not capable)– Instantaneous mission capability status at any time in the

simulation/analysis period

• System Level Performance Parameters– Average downtime per sortie– Average unscheduled maintenance time– Percent of scheduled sorties accomplished (over time)– Number of sorties cancelled due to pre-sortie failure– Number of unscheduled maintenance actions required

• Maintenance Resource Utilization Statistics– Total resource hours used during simulated period (by resource type)– Maximum number in use at any time during simulation– Total number of subsystem spare parts used

30


Characteristics of PC-based Modeling

• Can provide stochastic network processing with discrete events using simulation languages implemented on PC’s (SLAM II)

• Can simulate system operational environments:– Basic operations and maintenance processing defined by

established input networks– Specific task information (times, required resources, task

attributes, etc.) supplied through input data• Will treat system maintenance simulated at line replaceable

unit (LRU) level of detail with input and output data aggregated at the subsystem level of detail

• Provides real-time system capability assessment over a wide range of design and development parameters with relatively small set of input data required

• Use of real-time graphics capabilities promotes model understanding and display of results of different execution conditions and constraints

• Portability permits use in remote and dispersed locations for examining impacts of local environmental and support conditions

31


What Talents are Required?

• System Operators– Develop operational and support requirements and

concepts– Develop measures of effectiveness (MOEs) and

supportability (MOSs)

• System Modelers– Develop system-specific modeling and analysis

requirements, parameter definitions, input/output requirements

– Translate requirements into algorithmic definitions

• Applications Programmers– Implement the model(s) in the appropriate media solution

• Systems Analyst– Perform the required analyses and interpret results in

terms of system level impacts

32


Model System/Agency Applications Application/Purpose

Logistics/Composite Model (LCOM)

Aircraft:(C-17, B-1, F-11A,D,E,E-3A,F-4E,G,RF-E, A-10, A-7D, CH-53); Space Shuttle;Class.Sys.

Early Oper. Suitability Anal. Support Sys. Assessment Manpower Req. Analysis Mission Capab. Assessment

Avail./Readiness Model-Personal Comp.Application (ARM-PC)

C-17;FX-99;SABIR Satellite Systems (Mc Donnell Douglas, Rockwell Int’l)

Early Oper. Suitability Anal. Support Sys. Assessment Mission Capab. Assessment

General Workstation Analysis Model (GWAM)

Classified Systems/Clients Workst./Depot Flow Anal. Resource Req./CostProcess Throughput/Turnaround

(Missile) System Inventory Availability Model (SIAM)

Classified Systems/Clients Early Oper. Suitability Anal. Depot Req. AssessmentField/Deployed Avail. Anal.

Cruise Missile Availability Models (AAM and GAM)

Ground-Launched Cruise Missile (GLCM);Air-Launched Cruise Missile (ALCM)

Early Oper. Suitability Anal. Support Req. AssessmentField/Deployed Avail. Anal.

Availability-Oriented Provisioning Model (AOP)

Tracking/Data Relay Satellite Station (TDRSS);Class. Sys.

Spare Part Prov. Req.Optimum Spare Part Prov. For Availability

Logistics/Maint. Attack Model(LOGATAK,MACATAK)

Defense Nuclear Agency; US Army Logistics Center

Effects of Enemy Interdiction on Logistics Support Systems

Network Repair Level Analysis (NRLA) Model

Tactical Remote Sensor System (TRSS) Optimum Repair Level Analyses

Previous Availability Model Applications

33


Summary and Conclusion

34


The Benefits of Availability Modeling & Analysis

• Availability Modeling and Analysis provides the “glue” which ties system RMS performance evaluation together:– Considers operational environments/stresses– Identifies dominant failure modes’– Balances overall support system performance

• It provides one of the few methods capable of estimating fielded system performance levels during the design and development process.

• Applies to Commercial as well as DoD systems

35


Availability Analysis: A value-added process

• Availability analysis provides the “glue” which ties system RMS performance evaluation together:– Considers operational environments and

stresses– Identifies dominant failure modes– Incorporates repair and replace times

estimates– Evaluates overall support system

performance

36


Availability Analysis: A value-added process

• It provides a rational structure for evaluating system design and development decisions based on system level performance measures.

stracener_emis 7305/5305_spr08_04.17.08 1 system availability modeling dr. jerrell t. stracener, sae...

Documents