Sensitivity Analysis of the Scenario-Based

Reliability Estimation Algorithm

Artavazd Ghazaryan Michael Yetter

Independent Study Professor Judith Stafford

Tufts University

May 1, 2003

Abstract This paper explores the sensitivity analysis of component-based software systems. More

specifically, the research presented herein seeks to determine the reaction of overall

application reliability to change in individual component or transition reliabilities. The

paper also details the validation and implementation of the scenario-based reliability

estimation (SBRE) model and algorithm. Yacoub, Cukic, and Ammar originally

presented this model, which was used during this research to calculate the application

reliability, in their paper entitled “Scenario-Based Reliability Analysis of Component-

Based Software”[2].

Keywords

Scenario-Based Reliability Estimation, Scenario-Based Reliability Analysis, Sensitivity

Analysis, Component Dependency Graph

1. Introduction

Component-based software is a growing sector of software development that allows the

efficient creation of new applications by assembling existing software components

referred to as Commercial Off-The-Shelf (COTS) components, which are often well

defined and tested. Programming with COTS can result in major savings during

development, but also creates the problem of predicting how reliably component-based

systems will function once assembled.

- 1 -

One key problem rising from reliability prediction is sensitivity. Sensitivity seeks to

predict how the reliability of the application will be affected by variations in the

reliabilities of individual components or transitions. The experimentation and analysis

presented in this paper uses the SBRE reliability prediction model to explore sensitivity

in component-based applications.

1.1 Purpose

The purpose of this paper is to determine the relationship between the reliability of

component-based systems and variations in the reliabilities of its constituent components

in the hopes of furthering the ability programmers to create predictable software

assemblies. In addition to sensitivity analysis, the paper makes initial steps towards

validating the SBRE algorithm and suggests directions for future validation work.

1.2 Motivation

The Predictable Assembly from Certifiable Components (PACC) initiative of the

Software Engineering Institute (SEI) as detailed in the paper, “Compositional Reliability:

An Approach for PACC”[1] provides the motivation for the work detailed in this paper.

The PACC paper reviews several approaches to predicting application reliability and

outlines steps towards validating the techniques. Two of the suggested validation

procedures are manual validation and sensitivity analysis, both of which were applied to

the SBRE algorithm.

- 2 -

1.3 Why Choose the SBRE Approach?

The paper from the PACC initiative of the SEI[1] proposes that the SBRE approach

appears to be the best adapted to the goal of determining component-based application

reliability, citing the following reasons[1]:

• The approach assumes that source code for components is not available so, for example, fault injection and seeding are not applicable. This assumption meshes nicely with those of PACC.

• The approach is intended to apply during design, and does not rely on test cases or injecting faults into a running system or its environment. Again, this is a good fit for PACC.

• The CDG can probably be represented straightforwardly using the PIN model. • The approach is scenario-based, which takes into account the kind of usage-based

analysis intended to be covered by operational profiles.

2. Scenario-Based Reliability Estimation

2.1 The Model

The Scenario-Based Reliability Estimation model is a proposed method for determining

the reliability of a component based software application early in the design process,

prior to the actual assembly of the components. The SBRE model requires that a

component dependency graph (CDG) be used to represent a system. This CDG is

annotated with detailed information about the components, transitions, and the

application as a whole. The model then uses an algorithm to predict the reliability of the

application, Rappl. Each of the required annotations is described in the sections below.

2.1.1 AEappl – Average Execution Time of the Application

The average execution time of the application, AEappl, is amount of time, on average,

that is takes for the entire application to finish. The SBRE model primarily uses AEappl

- 3 -

to control the number of times that a loops execute, in order to prevent infinite loop

conditions. The average execution time of the application AEappl is given by:

∑=

=s

kkk STimePSAEappl

1)(* (1.1)

where PSk is probability of execution of scenario Sk; S is the total number of scenarios

and Time(Sk) is the average execution of scenario Sk

2.1.2 AECi – Average Execution Time of Component i

The average execution of each component, Ci, is simply the amount of time during which

that component executes. Ideally, the value of Ci can be obtained from a component’s

specification sheet; however, when a specification sheet is not available the value can be

determined by testing the component. The SBRE model sums the value of AECi for each

component as it traverses each branch of the CDG and causes the model to exit if the sum

exceeds AEappl. AEc is given by:

∑=

∈=s

kSkCiicici CTimePSAE

1)(* (1.2)

where PSk is probability of execution of scenario Sk; S is the total number of scenarios

and Time(Ci) is the average execution time of a component measured as the sum of its

active time from the given scenario sequence diagram.

2.1.3 RCi – Reliability of Component i

The reliability of each component, Ci, is defined as the probability that the component

will operate correctly when executed. As with AECi the value of RCi would ideally be

- 4 -

obtained from the component’s specification sheet, but in the absence of such a sheet,

should be determined by testing.

2.1.4 PTij – Transition Probability

The transition probability, PTij, is the probability that execution will follow the transition

from Ci to Cj. It can be estimated using the number of interactions between the two

components in the scenarios:

kji SCC

s

k Nlli

jikij CCInteract

CCInteractPSPT

∈= =∑ ⎟

⎟⎠

⎞⎜⎜⎝

⎛=

,1 ,..,1|),(||),(|

* (1.3)

where S is the number of scenarios; PSk is probability of execution of scenario Sk; N is

the number of components in the application and | is the number of

times C

),(| ji CCInteract

i interacts with Cj in a given scenario.

The second term in parenthesis is the probability of transition from Ci to Cj in a given

scenario that is then averaged and summed across all scenarios yielding the probability

for a given transition from Ci to Cj.

Furthermore, the SBRE model defines that the sum of the transition probabilities from

any component should be unity:

∑=

=N

jijPT

1

1 (1.4)

2.1.5 RTij – Transition Reliability

The reliability of the transition between Ci and Cj, RTij, is the probability that the

transition will execute successfully. The value of RTij should be obtained from the

- 5 -

specification sheet for that transition or through testing if a specification sheet is not

available.

2.2 The Proposed Algorithm

From the proposed model for scenario-based reliability estimation, Yacoub et. al. move

further to propose an algorithm that can automatically traverse the CDG to produce

Rappl.

2.2.1 Overview of How the Algorithm Works

The SBRE algorithm traverses the CDG using a technique similar to depth first search,

descending on a branch until either a terminating node is found or the sum of the average

execution times of the components along that branch exceed AEappl. As the algorithm

descends, it multiplies the reliabilities of the components and transitions as well as the

probabilities of transition along the path, storing the product in Rtemp. Once either of the

stop conditions is met, the algorithm returns to the last OR, adds the value of Rtemp to

Rappl and proceeds down the next branch. The algorithm is formally defined as follows:

Algorithm[2]

Procedure SBRE Parameters Consumes CDG, AEappl Produces Rappl Initialization: Rappl = 0, Time =0, Rtemp = 1 Algorithm push tuple <C1, RC1, AEC1>, Time, Rtemp while Stack not EMPTY do pop<Ci, RCi, AECi>, Time, Rtemp if Time > AEappl or Ci = t (terminating node) Rappl += Rtemp (OR path) else for each <Cj, RCj, AECj> that is a child of Ci push(<Cj, RCj, AECj>, Time += AECi, Rtemp = Rtemp*RCi*RTij*PTij) end end while

- 6 -

2.3 Manual Validation

An initial goal of our analysis of the SBRE algorithm was to manually verify its function

and accuracy; we were able to accomplish the first half of this goal. We verified the

functionality of the algorithm by generating a series of simple CDGs and found that

SBRE produced reasonable results for cases in which the CDG followed OR paths and

the sum of the transition probabilities from any one node was unity. However, we

determined that the algorithm would produce values of Rappl in excess of one for some

CDGs representing parallel systems such as that shown in section 3.2.3; an impossible

result. In these cases, we found that in order to represent a parallel system, the sum of the

transition probabilities would be greater than one for at least one component, a violation

of the rules of the SBRE model.

While we were able to determine types of CDGs for which the SBRE model produced

reasonable results, we could not be certain of the true accuracy of those results without

comparing our calculated reliability to those generated by extensive laboratory testing of

an application, a task that was outside of the feasible scope of our project. Given this

limitation, we concluded that the results produced by the algorithm for simple CDGs, if

not entirely accurate, would still be meaningful for the purpose of sensitivity testing.

2.4 Implementation of the Algorithm

To further explore the behaviors of the algorithm that we identified using manual

validation, we implemented a rudimentary software version of the algorithm in C++. See

Appendix A for the source code of this program, and Appendix B for the README file.

- 7 -

2.4.1 Structure

When designing our implementation of the SBRE algorithm, we closely followed the

structure of the proposed algorithm. In fact, the main() function of the program contains

all of the code that simulates the algorithm, the remaining functions support main by

creating data structures, consuming the input files, finding the correct nodes and

transitions, and generating output.

For the storage of each node, transition, and stack’s data, structs were defined in the

cdg.h header file, which is listed near the end of Appendix A. Arrays of structs were then

used to store all of the nodes and transitions for a given CDG. In addition, the list from

the C++ standard template library (STL) was used to store the transitions leaving each

node and the stack from the STL was used as the stack in the main algorithm.

Due to limited time and for the sake of simplicity during testing, we constructed the

program using fixed-size arrays for data storage rather than more complex structures such

as linked lists. However, for CDGs with more than ten nodes or transitions (the size

defined in the original version of the program), it is simple to edit the size of the fixed

arrays and then recompile the program. See the README file in Appendix B for further

details concerning modifications to the program.

2.4.2 Program Variables

The implementation of the program includes all of the variables defined in section 2.1 as

well as other variables which were used to contain transition, node, and stack data. The

additional variables are as follows:

- 8 -

node nodes[N] – This array of type node struct is used to store the data for up to N nodes. int nodecount – This variable must be initialized to the same N as nodes[N], and is used for loop control. transition trans[N] – This array of type transition struct is used to store the data for up to N transitions. int transcount - This variable must be initialized to the same N as trans[N] and is used for loop control. Stack<stackTuple> myStack – This STL stack of type stackTuple is used as the stack in the main algorithm. stackTuple temp1, temp2 – These stackTuples are used to temporaily store node data after nodes are popped off of the stack or before they are pushed. transition transtemp – This is used in the main algorithm to store the current transition that is returned by the call to get_trans().

2.4.3 Time and Space Complexity

While time limitations and project scope prohibited the steps necessary to formally define

the time and space complexity of the SBRE algorithm, observation showed that the

running time of the algorithm grew in linear fashion until AEappl exceeded the sum of all

AECi at which point growth stopped. For CDGs with loops, the running time grew more

than linearly with AEappl and the rate of increase grew with the number of loops in the

CDG. Space complexity was directly related to the total number of transitions and nodes.

2.4.4 Suggestions for Future Improvements

The current version of the SBRE program as presented in this paper was developed for

the purpose of limited testing and is not as robust or generalized as it should be for

ongoing use. If the program is to be used for continuing research, the following changes

would be useful:

- 9 -

• Additional Error Checking: The current program outputs an error message if

either of the input files is not present, but does not check the correctness of the

format or data content of the input files. It is recommended that the program be

modified to check the input files in order to catch errors before they cause

undesired, erroneous, or confusing outputs from the program.

• Generalized Node and Transition Storage: The fixed size arrays in the current

program should be replaced with dynamic data structures such as linked lists. As

an alternative, the program could be re-written in a language that allows for

variable array size, such as Java.

3. Sensitivity Analysis of the SBRE Algorithm

In this section we present the sensitivity analysis of SBRE algorithm. We studied the

behavior of the algorithm by varying the input parameters. We looked at the application

reliability as function of component reliabilities and as function of transition reliabilities.

3.2 Sensitivity with Simple CDGs

Once the program was implemented, we proceeded with sensitivity testing, beginning

with a few simple CDGs for which we manufactured reasonable data. We then varied

selected component reliabilities and transition probabilities and ran tests to determine the

effect of the changes on Rappl.

- 10 -

3.2.1 Basic OR Structure

The CDG in figure 1 below shows a basic OR structure in which control flows from

component C1 to either component C2 or C3 depending upon some condition. The graph

in figure 1 is used for tests one through three below.

T3

T1 T2

T4

T0

T

C2 C3

C1

S RC1 , AEC1 0.9 , 1.0

RC2 , AEC2 0.9 , 1.0 RC3 , AEC3 0.9 , 1.0 RT1 , PT1 1.0 , 0.5 RT2 , PT2 1.0, 0.5 RT3 , PT3 1.0 , 1.0 RT4 , PT4 1.0 , 1.0

Figure 1: CDG used for tests 1-3.

Test 1

Test 1 varied the reliability of component C3 from 0.0 to 1.0 by increments of 0.1. As

shown in Graph 1 of Appendix C, the value of Rappl grew linearly with the increased

reliability of C3.

Test 2

Test 2 set RC2 = .9 and RC3=.1, creating a situation where one branch of the OR structure

was highly unreliable. PT1 and PT2 were then varied from 1.0 and 0.0 respectively to 0.0

and 1.0 respectively by an increment of 0.1. As shown in Graph 2 of Appendix C, the

value of Rappl dropped linearly as the probability of transitioning to the branch

containing C3 was increased.

- 11 -

Test 3

Test 3 varied the reliability of transition T2 from 0.0 to 1.0 by increments of 0.1. As

shown in Graph 3 of Appendix C, the value of Rappl grew linearly with the increased

reliability of T2.

3.2.2 OR Structure with a Loop

The CDG in figure 2 below builds upon that of Figure 1 by adding T5 and creating a

loop. The CDG is used in tests four through six to determine the sensitivity of the

calculated value of Rappl to changes that occur within the loop.

T3

T1T2

T4

T0

T

C2 C3

C1

S

T5

RC1 , AEC1 0.9 , 1.0 RC2 , AEC2 0.9 , 1.0 RC3 , AEC3 0.9 , 1.0 RT1 , PT1 1.0 , 0.5 RT2 , PT2 1.0, 0.5 RT3 , PT3 1.0 , 1.0 RT4 , PT4 1.0 , 0.5 RT5 , PT5 1.0 , 0.5

Figure 2: CDG used for tests 4-6. Test 4 Test 4 fixed the reliabilities of all components at 0.9 and varied AEappl from 0.5 to 20.0

by increments of 0.5. As shown in Graph 4 of Appendix C, the value of Rappl decreased

asymptotically towards 0.76 as AEappl grew.

- 12 -

Test 5

Test 5 fixed RC3 at 1.0 and RT5 at 0.9, then varied AEappl from 0.5 to 20.0 by

increments of 0.5. As shown in Graph 5 of Appendix C, the value of Rappl decreased

asymptotically towards 0.78 as AEappl grew.

Test 6 Test 6 varied PT1 and PT2 from 1.0 and 0.0 respectively to 0.0 and 1.0 respectively by an

increment of 0.2 to change the extent to which the application favors the branch with the

loop. AEappl was then varied from 0.5 to 20.0 by increments of 0.5, creating a series of

Rappl data points for each pair of values PT1 and PT2. As shown in Graph 6 of Appendix

C, the value of Rappl decreased asymptotically for each series, with a lower asymptote

for PT1 and PT2 pairs that more heavily favor the looping branch.

3.2.3 A Parallel Structure

Figure 3 below is a CDG that represents a parallel structure, the PT0 – PT2 are each set to

1.0 to represent the behavior of the parallel system, which says that when the application

is at S, it has a 100% probability of following all three transitions.

T3

T1T2

T4

T0S

C1 C2 C3

TT5

RC1 , AEC1 0.9 , 1.0 RC2 , AEC2 0.9 , 1.0 RC3 , AEC3 0.9 , 1.0 RT1 , PT1 1.0 , 1.0 RT2 , PT2 1.0, 1.0 RT3 , PT3 1.0 , 1.0 RT4 , PT4 1.0 , 1.0 RT5 , PT5 1.0 , 1.0

Figure 3: CDG used for test 7.

- 13 -

Test 7

Test 7 seeks to explore the behavior of the application when the sum of the transition

probabilities from a given node exceeds 1.0. RC3 was then varied to illustrate the

algorithm’s reaction to change in reliability within a parallel system. As is shown in

Graph 7 of Appendix C, Rappl is equal to the sum of the reliabilities of the parallel

components and increases linearly with RC3.

3.3 Case Studies

In this section we apply the proposed algorithm to two simple component-based

applications implemented in object-oriented (OO) languages. For the purpose of this

paper, we are not concerned with “reusability” of the components. The applications are

independent from each other and the objects instantiated from the classes in each of the

programs are considered to be the runtime components. The Elevator Simulator is a very

simple program which models the basic operations of an elevator in which we deal with

two basic scenarios - passengers riding the elevator or waiting for the elevator to arrive.

The ATM Simulator models basic operations of an ATM such as a customer performing

banking transaction. The third is a GUI application that simulates a subway system.

Because of the large number of component interactions only a few of the use scenarios

are considered for the Subway Simulator.

First, we briefly describe the application architectures and their constituting components.

Then we construct the component dependency graphs (CDGs) based on the sequence

diagrams. We use the calculated parameters from the CDGs as the input to our SBRE

algorithm. The results of the analysis are discussed in the following section.

- 14 -

3.3.1 Architecture

Elevator Simulator

The architecture of Elevator Simulator application is centered around Elevator Control as

the main message dispatcher among the components. It is responsible for controlling the

other primary components Elevator Button, Door Button, Elevator and Door. These

objects are the run-time components implemented in object-oriented (OO) language. The

communication between the components (invocations) follows the form of method calls.

In our example, each of the components represents one class; in general the component

can have several classes.

The interaction between the components is analyzed using two basic use scenarios each

triggered by an event – the passenger pushes either the elevator button or the door button.

The sequence diagrams for these scenarios are shown in Appendix D.

• Riding the elevator – this scenario is executed when the passenger pushes the elevator

button. It describes the sequence of actions taken by the components to transport the

passenger in the elevator.

• Waiting for the elevator – this scenario is executed when the passenger pushes the

door button. It describes the sequence of action taken by the components in response

to a call from the door button.

Subway Simulator

The architecture of SuBway Simulator application is centered on Map object as the main

communication vehicle between the components in the scenarios. In addition, the other

primary components are Train, Track, Start/Stop Button, SlowDown Button, SpeedUp

Button, g2(Graphical Object), Line2D, Rectangle2D and timer.

- 15 -

As we noted earlier because of large number of interactions between components in this

example we limited the universe of possible scenarios to five. This is analogous to only

using the three controls of the application start/stop, slow down and speed up in addition

to the free simulation where the application animates the trains until stopped by the user.

The sequence diagrams for these scenarios are shown in Appendix D.

• Start the Simulation – This scenario executes when the start button is clicked. It

describes the sequence of action taken by the components to start up the timer and

execute one full cycle of animation

• Speed up - This scenario executes when the speed up button is clicked. It describes

the sequence of action taken by the components to increase the timer delay.

• Slow down - This scenario executes when the slow down button is clicked. It

describes the sequence of action taken by the components to decrease the timer delay.

• Animate (no user input) – This scenario executes after the start scenario was executed

with no other input from the user.

• Stop the Simulation - This scenario executes when the stop button is clicked. It

describes the sequence of action taken by the components to stop the timer.

- 16 -

3.3.2 The Component Dependency Graphs

Scenario Probability:

The probabilities of execution of the scenarios are assumed to be equal for both

applications. Hence, PSi=1/n, where n is the total number of use scenarios for a given

application.

Component and Transition Reliabilities:

We will not estimate the component and transition reliabilities; however we will use

these two parameters to discuss the sensitivity of the algorithm output - the application

reliability to the variations of component and transition reliability values.

Average Execution Time:

The average execution time of the application too will be used in analyzing the sensitivity

of the algorithm

Transition probabilities:

Based on the sequence diagrams for each scenario and the assumed probability of each

scenario the transition probabilities are calculated using Formula 1.3 on page 3. For this

paper we assumed normalizing constraints so that the summation of transition

probabilities for a given component to unity is satisfied. We suggest verifying this

technique in future work. With the parameters calculated above and the CDG tuplet

definition of the CGDs for both applications are constructed as shown in Figures 4 and 5.

- 17 -

S.5 .5

Door Button Elevator Button

1 1 .16.16

Elevator Control

.341 .34

Elevator Door

1 T

Figure 4 – Elevator CDG

- 18 -

S

.036

.4 .2

.2

4. Results an

4.1 Simple CD Tests one throu

application and

Additionally, te

transition to fav

Tests four throu

specifically, the

ButtonSS

.146

1.0

.2

.07

k

Map

1.0

1.0 1.0

.64

.066

.146

n

Timer

Trac

.218

.145.116

2

Figure 5 – Subway S

d Discussion

Gs

gh three illustrate the direct rela

the reliabilities of each of its co

st two shows the direct linear e

or a branch that is either more o

gh six show the effect that loop

effect of variations in the reliab

- 19

ButtonSU

1.0

.18

Rectangle2

imulation CDG

tionship between the rel

nstituent components an

ffect of changing the pro

r less reliable that does

s have on application re

ility of components or t

-

ButtonSD

Trai

.03

D

Line2D G

iabilit

d tran

babili

not co

liabili

ransit

t

t

y of an

sitions.

ty of

ntain a loop.

ty and more

ions within

loops. The most significant observation within these tests is the asymptotic nature of the

data, which represents the fact that as the average application time increases, the

reliability of branches with loops containing components or transitions that are not

completely reliable will go to zero, leaving only the reliability of the non-looping

branches.

Tests four and five show the equivalence of transition and component reliability values.

When correcting for minor differences in the input of these two tests, the reliability of the

application produced by both is same, even though the loop in test four contains a less

than reliable component versus a less than reliable transition in test five.

Test seven confirmed our suspicion that either the SBRE algorithm or the CDGs created

under the SBRE method are unable to correctly represent and determine the reliability of

applications with parallel components. Test seven shows calculated values of Rappl that

exceed one, a nonsensical result and statistical impossibility

4.2 Case Studies Using our implementation of SBRE algorithm we were able to study the application

reliability as function of individual component reliabilities. The graph in Figure 6 shows

the reliability of the application as function of reliability of one component at a time

while the reliabilities of other components are fixed to be equal to 1.

- 20 -

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

00.10.20.30.40.50.60.70.80.91

Re l i a bi l i t y of a Compone nt

Rappl (DoorBut t on)

Rappl (Elevat orBut t on)

Rappl (Elevat orCont rol)

Rappl (Elevat or)

Rappl (Door)

c

Figure 6 – Application reliability as function of component reliabilities (one at a time)

Component Reliability

Rappl (DoorButton)

Rappl (ElevatorButton)

Rappl (ElevatorControl)

Rappl (Elevator)

Rappl (Door)

0.9 0.912031 0.912033 0.771907 0.917829 0.912522 0.8 0.830143 0.830144 0.59366 0.845453 0.825045 0.7 0.753875 0.753873 0.454155 0.781714 0.737569 0.6 0.682798 0.682798 0.344196 0.725573 0.650093 0.5 0.616519 0.616519 0.256651 0.676082 0.562615 0.4 0.554668 0.554667 0.185827 0.632393 0.47514 0.3 0.496898 0.496896 0.127447 0.593747 0.387663 0.2 0.442891 0.442892 0.078378 0.559465 0.300187 0.1 0.39235 0.392349 0.0364038 0.528948 0.21271

Table 1: is the tabular representation of the graph in Figure 6.

From Figure 5, the application reliability varies significantly with the variation in the

reliability of the component Elevator Control. This is due to the fact that it is the main

communication vehicle in the assembly and it is executed more compared to the other

components.

- 21 -

5. Future Work

Recommendations for future work are as follows:

• Further explore how the SBRE algorithm or CDG representation can be modified

to correctly handle situations where the sum of the transition probabilities from a

given node is not unity, for example applications with parallel components

• Verify the method for calculating the transition probabilities based on the number

of interactions in analysis scenarios, for example in cases when one or more

components are not executed in some of the scenarios

• Conduct additional manual validation experiments in order to determine the true

validity of the application reliabilities generated by the SBRE algorithm.

6. Conclusion

This paper presents sensitivity analysis of component-based applications using the SBRE

algorithm to determine the reliability of the application. The analysis finds that the

reliability of applications is significantly more sensitive to variations in component and

transition reliability when those components or transitions are invoked frequently, usually

as a part of a loop structure. The analysis of the SBRE model and algorithm does not

prove the validity of the reliability values, but finds them adequate for the purposes of

sensitivity testing. The proposed method for calculating the transition probabilities needs

to be verified. Additionally, potential faults in SBRE model or algorithm are identified, in

particular the apparent inability of the model to correctly represent and handle

applications with parallel components.

- 22 -

7. References

[1] Clements, Paul, John McGregor, and Judith Stafford, “Compositional Reliability: An Approach for PACC” Whitepaper for PACCR, 2002.

[2] Yacoub, Sherif, Bojan Cukic, Hany Ammar. “Scenario-Based Reliability Estimation of Component-Based Software” in Proceedings of the Conference of Software Reliability Engineering (ISRE99), 1999.

- 23 -

Appendix A – SBRE Algorithm Implementation Source Code /* ** File: sbre.cpp ** Desc: This program implements the Scenario Based Reliability ** Estimation (SBRE) algorithm as presented by Sherif ** Yacoub, Bojan Cukie, and Hany Amar on page 11 of their ** paper "Scenario-Based Reliability Analysis of ** Component-Based Software" ** Auth: Michael Yetter, Artavazd Ghazaryan ** Date: April 14, 2003 */ // Includes #include <iostream> #include <fstream.h> #include <stack> #include "cdg.h" using namespace std; //Function Prototypes void init_CDG(); void output_dot(); node get_child_node(string); transition get_trans(string); // Global Variables float Rappl = .0; float AEappl = .0; // Declare array of type node in which to store nodes. // Change the size of the array to accomodate more nodes. int nodecount = 0; node nodes[10]; // Declare array of type transition in which to store transitions. // Change the size of the array to accomodate more transition trans[10]; int transcount = 0;

- 24 -

/********************************** ** Function main: ** ** Calls init_CDG, declares the ** ** stack, and runs the algorithm. ** **********************************/ int main() { // Initialize the CDG values init_CDG(); // Declare the Stack stack<stackTuple> myStack; stackTuple temp1; stackTuple temp2; transition transtemp; cout << "Enter AEappl: "; cin >> AEappl; while(AEappl != 0.0){ // Begin the algorithm Rappl = 0.0; temp1.c = nodes[0]; temp1.Rtemp = 1.0; temp1.Time = 0.0; myStack.push(temp1); while(!(myStack.empty())) { temp1.c = myStack.top().c; temp1.Rtemp = myStack.top().Rtemp; temp1.Time = myStack.top().Time; myStack.pop(); if((temp1.Time > AEappl) || (temp1.c.id == "t")) { Rappl += temp1.Rtemp; } else { while(!(temp1.c.transList.empty())) { transtemp = get_trans(temp1.c.transList.front()); temp1.c.transList.pop_front(); temp2.c = get_child_node(transtemp.childID);

- 25 -

temp2.Rtemp = (temp1.Rtemp * temp1.c.RC * transtemp.Rt * transtemp.Pt); temp2.Time = (temp1.Time + temp1.c.AEc); myStack.push(temp2); } } } cout << "Rappl = " << Rappl << endl; cout << "Enter AEappl or 0 if finished: "; cin >> AEappl; } // end while // Output the CDG as a dot file output_dot(); return 0; } // END MAIN /*************************************************** ** Function get_child_node: ** ** Takes the string id of the desired child node, ** ** finds the array index of the correct node and ** ** return the node. ** ***************************************************/ node get_child_node(string theID) { for(int i=0; i < nodecount; i++) { if(nodes[i].id == theID) { return nodes[i]; } } cout << "ERROR: Node with ID " << theID << " not found!" << endl; exit(0); } // END GET_CHILD_NODE

- 26 -

/*************************************************** ** Function get_trans: ** ** Takes the string id of the desired transition, ** ** finds the array index of the correct transition ** ** and returns the node. ** ***************************************************/ transition get_trans(string trans_ID) { for(int i=0; i < transcount; i++) { if(trans[i].id == trans_ID) { return trans[i]; } } cout << "ERROR: Node with transition" << trans_ID << " not found!" << endl; exit(0); } // END GET_TRANS /**************************************************** ** Function output_dot(): ** ** Creates a dot file for use by Graphviz, a graph ** ** drawing program, available from www.graphviz.org ** ****************************************************/ void output_dot() { ofstream dotfile("sbre.dot"); stackTuple tempA; stackTuple tempB; stack<stackTuple> dotStack; tempA.c = nodes[0]; dotStack.push(tempA); list<string>::iterator j; dotfile << "digraph G {" << endl << endl; for(int i=0; i < nodecount; i++) { for(j=nodes[i].transList.begin(); j != nodes[i].transList.end(); ++j) { dotfile << nodes[i].id << " -> " << (get_trans((*j))).childID;

- 27 -

dotfile << " [label = " << (get_trans((*j))).id << "]" << endl; } } dotfile << endl << "}"; dotfile.close(); } /********************************************************* ** Function init_CDG: ** ** Consumes the node and transition data from the files ** ** nodes.in and trans.in and inserts the data into ** ** the node and transition arrays respectively. ** *********************************************************/ void init_CDG() { // Needed Variables ifstream nodefile("nodes.in"); ifstream transfile("trans.in"); string trans_id; // Check for the absence of the input files if(nodefile == NULL) { cout << "ERROR: nodes.in not found" << endl; exit(0); } if(transfile == NULL) { cout << "ERROR: trans.in not found" << endl; exit(0); } // Consume Nodes Here while(!(nodefile.eof())) { nodefile >> nodes[nodecount].id; nodefile >> nodes[nodecount].RC; nodefile >> nodes[nodecount].AEc; for(int i = 0; i < 10; i++) { nodefile >> trans_id;

- 28 -

if(trans_id == ";") { break; } else { nodes[nodecount].transList.push_front(trans_id); } } nodecount++; } nodefile.close(); // Consume Transitions Here while(!(transfile.eof())) { transfile >> trans[transcount].id; transfile >> trans[transcount].childID; transfile >> trans[transcount].Rt; transfile >> trans[transcount].Pt; transcount++; } transfile.close(); } /* **File: cdg.h **Header File for sbre.cpp **Contains struct definitions */ #ifndef CDG_H #define CDG_H #include <iostream> #include <string> #include <list> struct transition {

- 29 -

string id; string childID; float Rt; float Pt; }; // Node Struct struct node { string id; float RC; float AEc; // list of transitions list<string> transList; }; struct stackTuple { node c; float Rtemp; float Time; }; #endif

- 30 -

Appendix B – Contents of the README File for the SBRE Program README File for sbre.cpp Author: Michael Yetter Date: April 14, 2003 Notes: sbre.cpp requires the file cdg.h for compilation. How to use sbre.cpp: 1) Write the two required input files: "nodes.in" : This file should contain the nodes in the component dependency graph (CDG) with the data for each node on its own line in the following format: [node name] RC AEc [outgoing transition name]* ; Where: [node name] is a string RC = Reliability of the Component as a float AEc = Averge Execution Time of the Component as a float [outgoing transition name]* = the string name of each outgoing transition. If there are no outgoing transitions then omit this element. ; Each line MUST be terminated by a semi-colon that is seperated from the preceding data element by one space. Example "nodes.in" contents: s 1.0 0.0 trans0 ; c1 .2 3.0 trans1 trans2 ; c2 .4 4.0 trans3 ; c3 .7 6.0 trans4 ; c4 .8 3.0 trans5 trans6 ; t 1.0 0.0 ; "trans.in" : This file should contain the transitions in the CDG with the data for each transition on its own line in the following format:

- 31 -

[transition name] [child name] Rt Pt Where: [transition name] is a string [child name] is the string name of the node that the transition points to. Rt = Reliability of the Transition as a float Pt = Probability of the Transition as a float Example "trans.in" contents: trans0 c1 1.0 1.0 trans1 c2 .8 1.0 trans2 c3 .2 1.0 trans3 c4 1.0 1.0 trans4 c4 1.0 .9 trans5 c3 .7 1.0 trans6 t .3 1.0 2) Run the program: a) Type in the name of the compiled program (ie a.out) b) The program prompts you to input the Average Execution Time of the Application (AEappl). Enter a value as a float and press enter. Continue entering AEappl until you are finished, then enter 0 or 0.0 to exit the program. 3) Convert the output file "sbre.dot" : To convert the dot output file into a postscript CDG graph type: dot -Tps sbre.dot -o sbre.ps You can then use ghostview to view the graph by typing: ghostview sbre.ps

- 32 -

Appendix C – Simple Tests: Graphical Results

Graph1: Rappl vs. RC

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 0.2 0.4 0.6 0.8 1 1.2

RC

Rap

pl

Graph 2: Rappl vs. PT

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 0.2 0.4 0.6 0.8 1 1.2

PT

Rap

pl

- 33 -

Graph 3 : Rappl vs. RT

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0 0.2 0.4 0.6 0.8 1 1.2

RT

Rap

pl

Graph 4 : RC3 = .9

0.74

0.76

0.78

0.8

0.82

0.84

0.86

0.88

0.9

0.92

0 5 10 15 20 25

AEappl

Rap

pl

- 34 -

Graph 5 : RT5 = .9

0.78

0.8

0.82

0.84

0.86

0.88

0.9

0.92

0 5 10 15 20 25

AEappl

Rap

pl

Rappl

Graph 6 : AEappl vs. Rappl

0.6

0.65

0.7

0.75

0.8

0.85

0 5 10 15 20 25

AEappl

Rap

pl

PT = 0.0PT = 0.2PT = 0.4PT = 0.6PT = 0.8PT = 1.0

- 35 -

Graph 7 : Parallel Components

0

0.5

1

1.5

2

2.5

3

0 0.2 0.4 0.6 0.8 1 1.2

RC3

Rap

pl

- 36 -

Appendix D - Sequence Diagrams for Analysis Scenarios

Elevator Sim

: Passenger Elevator Button Elevator Control Elevator Door

1: press button2: update

3: illuminate4: reach

5: floor_reach

6: stop7: cancel_illuminate

8: open

9: close

Figure 1: “Riding the elevator” scenario.

: Passenger Door Button Elevator Control Elevator Door

1: press button2: update

3: illuminate4: move

5: floor_reach

7: cancel_illuminate6: stop

8: open

9: close

Figure 2: “Waiting for the elevator” scenario.

- 37 -

Subway Sim

: User Start Button : ButtonSS

Map : Map timer : (Logical View::javax:...

Train : RegT Track : LineTrack

g2 : (Logical View::java::a...

Line2D : (Logic...

Rectangle2D

1: Start Simulation2: StartSim()

3: start()

4: tick()5: getY2()

7: getY1()

9: getX2()

11: getX1()

13: getX()

14: getSpeed()

15: setX()

16: getY()

17: setY()

18: drawBC()

19: drawTD()

20: draw()

23: gettheColor()

24: setPaint()

25: getX1()

26: getY1()

27: getX2()

28: getY2()29: new

32: getName()

31: draw()

33: drawString()

6: Y2

8: Y1

10: X2

12: X1

34: draw()

37: getSH()

30: line

21: new

22: g2

35: new

36: g2

38: gettheColor()

39: setPaint()

40: getX()

41: getY()

42: new

43: loc

44: new

45: frame

46: fill()

47: getName()

48: drawString()

49: getSpeed()

50: drawString()

51: setPaint()

52: draw()

Figure 3: “Start the simulation” scenario.

- 38 -

: User Speed Up Button : ButtonSU

map : Map timer : (Logical View::javax:...

1: Speed Up2: setSimSpeed()

3: setDelay()

Figure 4: “Speed Up” scenario.

: User Slow Down Button : ButtonSD

map : Map

1: Slow Down2: setSimSpeed()

timer : (Logical View::javax:...

3: setDelay()

Figure 5: “Slow Down” scenario.

- 39 -

Map : Map Train : RegT Track : LineTrack

g2 : (Logical View::java::a...

Line2D : (Logic...

Rectangle2D

15: drawBC()

16: drawTD()

1: tick()

10: getX()

11: getSpeed()

44: getName()

46: getSpeed()

12: setX()

13: getY()

38: getY()

14: setY()

35: gettheColor()

37: getX()

34: getSH()

31: draw()

2: getY2()

4: getY1()

6: getX2()

8: getX1()

3: Y2

5: Y1

7: X2

9: X1

17: draw()

20: gettheColor()

22: getX1()

23: getY1()

24: getX2()

25: getY2()

29: getName()

21: setPaint()

28: draw()

30: drawString()

18: new

19: g2

32: new

33: g2

36: setPaint()

43: fill()

45: drawString()

47: drawString()

48: setPaint()

49: draw()

26: new

27: line

39: new

40: loc

41: new

42: frame

Figure 6: “Animate” scenario.

- 40 -

: User Stop Button : ButtonSS

Map : Map timer : (Logical View::javax:...

1: Stop Simulation2: StopSim()

3: stop()

Figure 7: “Stop” scenario.

- 41 -