International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013 1

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Testing-Effort Dependent Software Reliability Model for Distributed Systems

Omar Shatnawi, Department of Computer Science, Al al-Bayt University, Mafraq, Jordan

ABSTRACTDistributed systems are being developed in the context of the client-server architecture. Client-server archi-tectures dominate the landscape of computer-based systems. Client-server systems are developed using the classical software engineering activities. Developing distributed systems is an activity that consumes time and resources. Even if the degree of automation of software development activities increased, resources are an important limitation. Reusability is widely believed to be a key direction to improving software development productivity and quality. Software metrics are needed to identify the place where resources are needed; they are an extremely important source of information for decision making. In this paper, an attempt has been made to describe the relationship between the calendar time, the fault removal process and the testing-effort consumption in a distributed development environment. Software fault removal phenomena and testing-effort expenditures are described by a non-homogenous Poisson process (NHPP) and testing-effort curves respec-tively. Actual software reliability data cited in literature have been used to demonstrate the proposed model. The results are fairly encouraging.

Keywords: Distributed Development Environment, Fault Severity, Software Component, Software Engineering, Software Reliability, Software Testing-Effort

INTRODUCTION

The software development environment is changing from a host-concentrated one to a distributed one due to cost and quality as-pects and rapid growth in network computing technologies. Distributed systems are being developed in the context of the client-server architecture. Client-server architectures domi-nate the landscape of computer-based systems. Everything from automatic teller networks to the Internet exists because software residing

on one computer—the client—requests ser-vices and/or data from another computer—the server. Client-server software engineering blends conventional principles, concepts, and methods with element of object-oriented and computer-based software engineering to create client-server systems. Client-server systems are developed using the classical software engineering activities.

Distributed systems are growing rapidly in response to the improvement of computer hardware and software and this is matched by

DOI: 10.4018/jdst.2013040101

2 International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

the evolution of the technologies involved (Zhao et al., 2010). Developing large-scale distributed systems (LSDS) is complex, time-consuming, and expensive. LSDS developments are now common in air traffic control, telecommuni-cations, defense and space. In these systems a release will often be developed over 2-4 years and cost in excess of 200-300 person years of development effort (Kapur et al., 2004b). Due to their complexity, LSDS are hardly ever “perfect” (Lavinia et al., 2011). Features define the content of a release. The features are realized by map-ping the full set of system feature requirements across the various components. Normally this involves building on a large existing software base made up of the existing components that are then modified and extended to engineer the new release. Requirement changes are common in these developments because of the long lead times involved. Change is inevitable since user requirements, component interfaces, and developers’ understanding of their application domain all change. This adds to the complex-ity of planning and controlling the in-progress development both at the individual component and the release. A further complexity factor is the integration and validation of the release once the individual components are delivered. These large-scale systems must achieve high reliability because of their nature. In the final release validation software defects arise not only from the new software but also due to latent defects in the existing code. Substantial regres-sion tests must be run to checkout the existing software base that may amount to millions of lines of code (Kapur et al., 2004b).

Successful operation of any computer sys-tem depends largely on its software components. Thus, it is very important to ensure the quality of the underlying software in the sense that it performs its functions that it is designed and built for. To express the quality of the software to the end users, some objective attributes such as reliability and availability should be measured. Software reliability is the most dynamic qual-ity attribute (metric), which can measure and predict the operational quality of the software. Software reliability model (SRM) is the tool, which can be used to evaluate the software

quantitatively, develop test cases, schedule status and monitor the change in reliability performance. In particular, SRMs that describe software failure occurrence or fault removal phenomenon in the system testing phase are called software reliability growth models (SRGMs). Among others, non-homogeneous Poisson process (NHPP) models can be easily applied in the actual software development.

Some SRGMs are concerned with the cu-mulative number of faults detected by software testing and the testing period (Lyu, 1996; Xie, 1991; Musa, 1999; Kapur et al., 1999; Pham, 2000; Shatnawi, 2009a). These models assume testing-effort to be constant throughout the test-ing period. Other SRGMs have incorporated the expenditures due to testing-effort (Yamada et al., 1985; Kuo et al., 2001; Kapur & Bardhan, 2002; Kapur et al., 2006; Huang et al., 2007; Kapur et al., 2008). They assumed that the fault detection rate is proportional to current fault-content and software testing-effort expenditures. Testing-effort expenditures are the resources spent on software testing. It has been observed that the relationship between the testing time and the corresponding number of faults removed is either Exponential or S-shaped. An interesting inference can be made regarding the analysis namely all the models are robust and can be used for any testing environment and can be termed as Black-Box models, which are used without having any information about the nature of the software being tested. However, if one has to develop what is called White-Box model, one needs to know about the software technology, which has been used to develop the software. Thus, it is imperative to clearly understand the software development environment and accord-ingly there is need to develop a model, which can explicitly explain the software technology that has been used to develop the software and now being tested. Such a modelling approach was earlier adopted by (Pham, 2000; Shatnawi, 2009b; Kapur et al., 2004a; Ohba, 1984; Yamada et al., 1985). Such approach is very much suited for object-oriented programming and distrib-uted development environments (Kapur et al., 2004a; Shatnawi, 2009b).

International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013 3

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Considerable evidence from industry case studies indicates substantial business benefit can be derived from aggressive software reuse. In ideal setting, a software component that is developed for reuse would be verified to be correct and would contain no defect. In reality, formal verification is not carried out routinely, and defects can and do occur. However, with each reuse, defects are found and eliminated, and a component’s quality improves as a result. In a study conducted at Hewlett-Packard (HP), Lim (1994) reports that the defect rate for reused code is 0.9 defects per KLOC, while the rate for newly developed software is 4.1 defects per KLOC.

The present scenario of software develop-ment life cycle has emerged into a distributed environment because of the development of network technology and ever increased demand of sharing the resources to optimize the cost. A software development process typically con-sists of four phases: specification, designing, coding, and testing (Ohtera & Yamada, 1990). During the testing phase, a lot of development resources are consumed to detect and correct faults latent in the developed software system. Testing distributed systems is an example of an activity that consumes time and resources (Cristescu & Ciovica, 2010). Therefore, an SRGM is needed to estimate the current reli-ability level and the time and resources required to achieve the objective reliability level. The proposed NHPP based SRGM attempts to ac-count for the relationship between the amount of testing-effort and the number of software faults detected during testing in distributed development environment. The proposed model is based on the assumption that the software system is composed of a finite number of reused and newly developed components. The reused components do not consider the effect of the impact (severity) of the fault-type (complexity) on the software reliability growth phenomenon (i.e., the growth is uniform). The newly devel-oped components do consider the effect of the impact of the fault-type on the software reli-ability growth phenomenon. Accordingly, the fault removal process is modelled separately and the total fault removal phenomenon is the

sum of the fault removal process of all. Then, we review some forms of testing-effort func-tions such as exponential, Rayleigh, Weibull, logistic etc. After that, we provide the methods used for parameter estimation and the criteria used for validation and evaluation respectively. We then validate and compare the model with other existing model by applying them on actual software reliability dataset cited from real software development projects. Finally, conclusions are drawn.

SOFTWARE RELIABILITY MODELLING

Computer software traditionally ran in stand-alone systems, where the user interface, ap-plication ‘business’ processing, and persistent data resided in one computer, with peripherals attached to it by buses or cables. Few interesting systems, however, are still designed this way. Instead, most computer software today runs in distributed systems, where the interactive presentation, application business processing, and data resources reside in loosely-coupled computing nodes and service tiers connected together by networks (Buschmann et al., 2007). Therefore, the software development environ-ment is changing from a host-concentrated one to a distributed one due to cost and quality aspects and rapid growth in network computing technologies. A distributed system is a comput-ing system in which a number of components cooperate by communicating over a network. The explosive growth of the Internet and the World Wide Web in the mid-1990’s moved distributed systems beyond their traditional application areas, such as industrial automa-tion, defense, and telecommunication, and into nearly all domains, including e-commerce, financial services, health care, government, and entertainment (Buschmann et al., 2007). Software components also has failure rate specified during software design using software engineering paradigm. These failure rates reflect the reliability of the system, which is desired to be high (Raza & Vidyarthi, 2011). Under this environment software modules/components can

4 International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

be developed at different geographical locations and components used in other software can be re-used. It is empirically known that the growth curve of the cumulative number of detected faults is an exponential curve when a software system consisting of several software com-ponents are tested in the testing phase (Ohba, 1984), while is an S-shaped growth curve when a newly developed software component is tested (Yamada et al., 2000).

For software systems developed under such an environment several models have been pro-posed (Yamada et al., 2000; Kapur et al., 2004b; Shatnawi, 2007). Yamada et al. (2000) have constructed a software reliability model based on an NHPP, which incorporates the exponential software reliability model (Goel & Okumoto, 1979), and the delayed S-shaped model (Yamada et al., 1983) for software systems developed under such an environment. They assume that a software system consisting of several number of reused software components and newly developed software components. Un-der these assumptions they formulate a model based on NHPP for a distributed development environment. The formulated model was found to describe only both the purely exponential growth curve and the highly S-shaped growth curve according to the value of the weight parameters. Though the authors of the model argue that if we can estimate the values of the weight parameters reasonably, we can obtain software reliability assessments measures more accurately, they have assumed prespecified testing-weight parameters for each software component. In addition, they assumed that as the testing time increases, testing-effort also increases. If testing time becomes quite large, testing-effort also becomes quite large. In real-ity, no software developer may spend infinite resources on testing irrespective of the testing period. Thus, there is a great need to develop more realistic software reliability model.

To address these issues, we integrate testing-effort function into Yamada et al. (2000) model to get a better description of the software fault removal phenomenon in a dis-tributed development environment. Different from Yamada et al. (2000) model, the proposed

model incorporates the effect of testing-effort and the values of the testing-weight parameters are estimated for capturing a wide class of reliability growth curves. The model consid-ers that software consists of a finite number of reused and newly developed components. Each component has its own characteristics and thus the faults detected in a particular component have their own peculiarities. Therefore, the fault removal process for each component can be modeled separately and the total fault removal phenomenon is the sum of the fault removal process of all the components. We feel the problem can be better formulated using the proposed model developed below.

Model Assumptions

1. Fault removal phenomena follows an NHPP with mean value function m(t).

2. Software is subject to failures during execution at random times caused by the manifestation of remaining faults in the system.

3. Software is composed of a finite number of newly developed and reused components. Accordingly, faults are of two types, which are of different severity. Testing effort required depends on the severity of faults.

4. From our discussion in the introduction it follows that the ratio of fault exists in reused to newly developed components is about 1 to 4.

5. Software reliability growth in the reused components is uniform while in the newly developed component is not.

6. Each time a failure occurs, an immediate (delayed) effort takes place to decide the cause of the failure in order to remove it. The time delay between the failure obser-vation and its subsequent fault removal is assumed to represent the severity of the faults.

7. Fault removal process (i.e., the debugging process) is perfect.

8. Mean number of faults detected/removed by the current testing-effort expenditures is proportional to the mean number of remaining faults in the system.

International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013 5

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Model Notations

a: Over-all fault-content of the software; (∑ai+∑aj=a), a>0

ai: Initial fault-content of fault-type i; (i=1,2,…,m), ai(=ahi)

aj: Initial fault-content of fault-type j; (j=1,2,…,n),aj(=ahj)

h i ( j ) : Propor t ion of faul t - type i ( j ) ; (0<hi≤.2,0<hj≤.8,∑hi+∑hj= 1)

W(t): Over-all testing-effort consumption by time t; (∑Wi(t)+∑Wj(t)=W(t)), W(t)=∫w(x).dx

Wi(j)(t): Cumulative effort spent on removal of fault-type i(j);Wi(t)(=giW(t)), Wj(t)(=gjW(t))

wi(j)(t): Current testing-effort spent on removal of fault-type i(j) at time t

gi(j): Proportion of resource consumption i(j); (0<gi≤.2,0<gj≤.8,∑gi+∑gj= 1)

bi(j): Fault detection rate per unit testing-effort per fault-type i(j)

bjf(r): Fault detection/removal rate per unit testing-effort per fault-type i(j)

bir: Fault removal rate per unit testing-effort per fault-type j

mjf(t): Mean number of failures caused by fault-type i(j) by time t

mi(j)r(t): Mean number of fault removed of fault-type i(j) by time t

i, j: Subscripts that denotes the reused and newly developed components

c(t): Time dependent rate at which testing resources are consumed, with respect to remaining available resources

α,c,r: Constant parameter in testing-effort functions.

Modelling the Total Fault Removal Phenomena of ‘i’ Reused Components

To model the fault removal process of ‘i’ re-used components, the exponential model with testing-effort (Yamada et al., 1985) is adopted. The adopted model assumed that the faults in the reused components are simple faults,

which can be removed instantly as soon as they are observed. Hence, fault removal of reused components is modeled as one-stage processes.

Assuming that the mean number of faults removed in the time interval (t, t+∆t) by the current testing-effort is proportional to the mean number of faults not removed, this model can be expressed by the following differential equation

d

dtm t w t b a m tir i i i ir( ) ( ) ( )= −( ) (1)

Solving the above differential (1) under the boundary condition mir(t=0)=Wi(t=0)=0 yields

m t a eir i

bW ti i( ) ( ))= −( )−1 (2)

Modelling the Total Fault Removal Phenomena of ‘j’ Newly Developed Components

To model the fault removal process of of ‘j’ newly developed components, the delayed S-shaped model with testing-effort (Kapur & Bardhan, 2002; Huang et al., 2007) is adopted. The adopted model assumed that the faults of newly developed components consume more testing effort when compared with faults of reused component. This means that the testing team will have to spend more time to analyze the cause of the failure and therefore requires greater efforts to remove them. Hence the re-moval process for such faults is modeled as a two-stage process.

Assuming that the mean number of faults detected in the time interval (t, t+∆t) by the cur-rent testing-effort is proportional to the mean number of faults not detected, and the mean number of fault removed in the time interval (t, t+∆t) by the current testing-effort is proportional to the current number of faults not removed in the system, this model can be expressed by the following system of differential equation

6 International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

d

dtm t w t b a m t

d

dtm t w t b m t m

jf j jf j jf

jr j jr jf jr

( ) ( ) ( )

( ) ( ) ( ) (

= −( )= − tt)( )

(3)

Solving the above system of differential (3) under the boundary condition mjf(t=0)= Wj(t=0)=mjr(t=0)=0 yields

m t ab b

b e b ejr j

jr jfir

b W t

if

b W tjf j jr j( )( ) ( )= −

−−( )

− −1

1

(4)

Assuming the fault identification rate is approximately the same as the fault removal rate, and using L’Hospital’s rule (Huang et al., 2007), (4) can be rewritten as

m t a bW t ejr j j j

bW tj j( ) ( ( ))( ))= − +( )−

1 1 (5)

Modelling the Total Fault Removal Phenomenon

The proposed SRGM for distributed environ-ment is the superposition of the two NHPP of `i’ reused and j’ newly developed components with mean value functions given in (2) and (5) respectively. Thus, the mean value function of the superposed NHPP is

m t m t m t

a e a

iri

m

jrj m

n

ibW t

i

m

ji i

( ) ( ) ( )

(( )

= +

= −( )+ −

= = +

−

=

∑ ∑

∑1 1

1

1 1 1++( )−

= +∑ bW t ej j

bW t

j m

nj j( ))( )

1

(6)

TESTING-EFFORT MODELLING

Testing phase in the software development process aims at detecting and removing faults, and hence making the software more reliable. The testing phase, which aims to improve the reliability of a software system, is the most costly, time-consuming phase among the four phases. About half of the resources consumed during the software development cycle are

testing resources (Ohtera & Yamada, 1990). Moreover, because the sizes of software systems have increased significantly during the past decades, effective utilization of limited testing resource has become even more important than before (Wang et al., 2010).

Software testing-effort is measured by resources such as man power spent during test-ing, CPU hours, number of test cases etc. These testing resources spent in testing appreciably affect software reliability. The consumption curve of these resources over the testing period can be thought of a testing-effort curve. Various forms of testing-effort functions have been used in the literature such as exponential, Rayleigh, Weibull, logistic etc. (Yamada et al., 1985; Kuo et al., 2001; Kapur & Bardhan, 2002; Kapur et al., 2006; Huang et al., 2007; Kapur et al., 2008).

To study the testing-effort process, one of the below functions can be selected. The most frequently used function to explain the testing-effort are:

• Exponential• Rayleigh• Weibull• Logistic

The first two can be derived from the assumption that, “the testing effort rate is pro-portional to the testing resources available”.

d

dtW t c t W t( ) ( ) ( )= −( )α (7)

Solving (8) under initial condition W(t=0)=0, yields

W t e c x dxt

( ) ( )= −( )∫α 1 0 (8)

Case 1: When c(t)=c, we get exponential curve:

W t e ct( )= −( )−α 1 (9)

International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013 7

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Case 2: If c(t)=c.t, we get Rayleigh type curve:

W t e ct( ) /= −( )−α 12 2 (10)

Case 3: If c(t)=c.r.tr-1, we get Weibull function:

W t e ctr

( )= −( )−α 1 (11)

Exponential and Rayleigh curves become special cases of the Weibull curve for r=1 and r=2 respectively.

Case 4: If we define

d

dtW t c

W tW t( )

( )( )= −( )

αα (12)

On solving, the cumulative testing effort consumed in the interval (0, t) is given by

W tre ct

( )=+ −

α1

(13)

This is the Logistic testing-effort function.

PARAMETER ESTIMATION

Parameters estimation is of primary concern in software reliability prediction. Testing-effort data of a software can be collected in the form of testing-effort wk(0<w1<w2<…<wn) consumed in (0, ti) where i=1,2,..,n. The parameters (α>0, 0<c<1, r≥0) in the testing-effort functions are estimated by the method of least squares as follows

Minimize W W

Subject to W W

ii

n

n n

−( )==∑ ˆ

ˆ

2

1 (14)

where W Wn n= implies that the estimated

value of the testing-effort is equal to the actual value.

Using these estimated parameters values, we estimate the parameters in the proposed model given in (6) by the method of maximum likelihood estimation (MLE). The data are given in the form of cumulative number of faults removed, yi, in time (0, ti). The Likelihood function L for the unknown parameters with the mean value function m(t) takes on the form

L parameter W x

m t m t

x x

i i

i i

x x

i ii

i i

| ( , )

( ) ( )

( )!

( ) =−

−−

−

−

−

1

1

1

==

− −( )∏ −

1

1

nm t m t

e i i( ( ) ( ) (15)

Taking natural logarithm of (15) we get

ln

( )ln ( ) ( )

( ) ( ) ln (

L

x x m t m t

m t m t

i ii

n

i i

i i

=

− −

−

−( )

−=

−

−

∑ 11

1

1xx xi i

i

k

−

−

=∑ 11

)

(16)

The MLE of the parameters can be obtained to by maximizing ‘L’ with respect to the fol-lowing parameters constraints: (a>0, 0<bi(j)<1, 0<hi(gi)≤ 0.2, 0<hi(gj)≤ 0.8).

For faster and accurate calculations, the statistical package for social sciences (SPSS) based on the nonlinear regression technique has been utilized for the purpose.

MODEL VALIDATION AND EVALUATION

For application to actual software reliability failures data, and comparison of our proposed model with the previous model, we analyze a data set, PL/I application program test data (Ohba, 1984) tabulated in Table 1. The sized of the software system is approximately 1,317,000 lines of code (LOC). Over the course of 19 weeks, 47.65 CPU hours were consumed, and 328 software faults were removed and further studied in (Yamada et al., 2000; Kapur & Bard-han, 2002; Shatnawi, 2007; Huang et al., 2007).

8 International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Therefore, the data set allows direct comparison with the work of others. In this paper, we assume that this data set was observed from the testing phase after confirmation of the integration of all software components.

We evaluate the performance of the testing-effort functions using Bias, Variation, and RMSPE metrics. For Bias, Variation, and RM-SPE, the smaller the metric value the better.

• Bias: The difference between the observa-tion and prediction of number of failures at any instant of time i is known as PEi.(prediction error). The average of PEs is known as bias.

BiasPE

ki

i

k

==∑1

(17)

where k is the number of observations.

• Variation: The standard deviation of pre-diction error is known as variation.

Variationk

PE Biasii

k=

−( ) −( )=∑11

2

1

(18)

• Root Mean Square Prediction Error (RMSPE): It is a measure of closeness with which a testing-effort function predicts the observation.

RMSPE Bias Variation= +( )2 2 (19)

Next, we evaluate the performance of the models under comparison in terms of goodness of fit using SSE metric. For SSE, the smaller the metric value the better the model fits rela-tive to other models run on the same data set.

• The Sum of Squared Error (SSE): This metric measures the distance of a model estimate value from the actual data, as follows

SSE m t xi i

i

k

= −( )=∑ ( )

2

1

(20)

where m(ti) is the estimated cumulative number of faults by time ti obtained from the fitted mean value function given in (6) and xi is the total number of faults removed by time ti.

Table 1. PL/I software Failure Data

Test time

(week)

Cumulative execution time

(CPU hour)

Cumulative faults

Test time (week)

Cumulative execution time

(CPU hour)

Cumulative faults

1 2.45 15 11 26.23 233

2 4.9 44 12 27.67 255

3 6.86 66 13 30.93 276

4 7.84 103 14 34.77 298

5 9.52 105 15 38.61 304

6 12.89 110 16 40.91 311

7 17.1 146 17 42.67 320

8 20.47 175 18 44.66 325

9 21.43 179 19 47.65 328

10 23.35 206

International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013 9

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Lower value of SSE indicates less fitting error, thus better goodness of fit.

DATA ANALYSES AND MODEL COMPARISONS

For model validation and evaluation, we con-sider a simple case in which the software system composed of two reused software components and two newly developed. The proposed model given in (6) can be re-written as presented in Box 1.

The resultant parameters estimation and the performance of the testing-effort functions under comparison are tabulated in Table 2. From Table 2 we can see that the logistic testing-effort function has lower bias, variation, and RMSPE metric values. Therefore, the logistic testing-effort function gives the best estimated values among the functions under comparison. Note that the exponential testing-effort function fails to give any plausible estimation results.

The fitting of the testing-effort functions under comparison to the actual non-cumulative and cumulative testing-effort are graphically illustrated in Figures 1 and Figure 2 respec-tively. From Figures 1 and Figure 2, we can observe that the logistic testing-effort function provides a better fit than the other functions under comparison. Therefore, the logistic testing-effort function provides more accurate

description of resource consumption than other functions.

For model parameter estimation we have used logistic function as it gives best results as compared to other two effort functions (as seen in Table 2). It has been observed that the pro-portion parameters (h1, h2, h3, h4) and (g1, g2, g3, g4) values of the proposed model are (.15, .05, .35, .45) and (.19, .01, .39, .41) respec-tively. Yamada et al. (2000) assumed these values of (h1, h2, h3, h4) to be (.05, .05, .45, .45). The resultant parameters estimation and the performance of the models under comparison are tabulated in Table 3. Here it can also be seen that b1,2 > b3,4 as the testing teams have to spend more time to analyze and remove the cause of failure of fault-type (j=3,4) and there-fore require greater efforts to remove them as the faults in the components comprising a complete software can be of different severity. From Table 3 we can see that the proposed model has lower SSE metric value. Therefore, the proposed model fits better than Yamada et al. (2000) model.

Based on these estimated values the fault-content of each component and the amount of testing-effort for each component are tabulated in Table 4. These kinds of information pro-vided by the proposed model as shown in Table 4 may enable manager to decide how better he will allocate the resources so that testing time,

Box 1.

m t m t m t

a e a b

iri

jrj

ibW t

ij

i i

( ) ( ) ( )

(( )

= +

= −( )+ − +

= =

−

=

∑ ∑

∑1

2

3

4

1

2

1 1 1jj j

bW t

j

W t e j j( ))( )−

=( )∑

3

4 ( 2 1 )

where a ah a ah a h a ah a ah a h

a a

bkk

1 1 2 2 1 3 3 4 4 3

1

4

1

0 2 0 8= = = − = = = −

==∑, ( . ), , ( . )

== == = = − =b b b

W t W t g W t W t g W t g W t W t2 3 4

1 1 2 2 1 30 2

,

( ) ( ) , ( ) ( ) ( )( . ), ( ) ( )) , ( ) ( ) ( )( . )

( ) ( )

g W t W t g W t g

W t W tkk

3 4 4 3

1

4

0 8= = −

=

=∑

10 International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Table 2. Parameter estimation and comparison criteria metrics results

Testing-Effort Functions Under Comparison

Parameter Estimation

Comparison Criteria

α c r Bias Variation RMSPE

Exponential * * — * * *

Rayleigh 49.32 0.014 — -0.560 2.28 2.35

Weibull 799.22 0.002 1.115 -3.36 2.14 3.99

Logistic 54.84 0.226 13.03 0.062 1.31 1.33

* the function model fails to give any plausible result—the component is not part of the corresponding function

Figure 1. Non-cumulative testing-effort curves

Figure 2. Cumulative testing-effort curves

International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013 11

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

testing-effort and budget is optimized, quality is improved, and reduce maintenance efforts.

The fitting of the models under comparison to the actual non-cumulative and cumulative number of faults are graphically illustrated in Figures 3 and Figure 4. From Figure 4 we can observe that the estimated reliability growth curve is S-shaped. The cause of S-shapedness is attributed to the time delay between the fault removal and the initial failure observation, which may result of the unskilledness of the testing team. Therefore newly developed com-ponents have been tested more than the reused ones. That is why more faults found and more resources consumed in the newly developed components than reused one (as seen in Table 4). From Figures 3 and Figure 4 we can observe that the proposed model provides better descrip-tive than Yamada et al. (2000) model.

Based on our data analyses and model comparisons. We may conclude that incorporat-ing logistic testing-effort function into Yamada et al. (2000) model yields better results. There-fore, it is more appropriate to measure the software reliability growth phenomenon in distributed development environment with

respect to the amount of testing resources spent on testing.

CONCLUSION

The importance of research in software reli-ability and testing is not surprising due to the continuing growth in the software industry. Distributed systems are the foundation of the next-generation communications systems. Developing distributed systems is an activity that consumes time and resources. Testing distributed systems is an example of an activity that consumes time and resources. Therefore, an SRGM is needed to estimate the current reliability level and the time and resources re-quired to achieve the objective reliability level. In this paper a new SRGM based on NHPP for distributed development environment has been proposed. The model considers that software system consists of a several number of reused and newly developed components. The model takes into account severity of the faults, the time lag between the failure or fault detected and the fault removal processes in the newly developed components and also incorporates the effect of testing-effort consumptions which is described

Table 3. Parameter estimation and comparison criteria metric results

Models under Comparison

Parameter Estimation

Comparison Criteria

a b1,2 b3,4 SSE

Yamada et al. (2000) Model 378.12 0.465 0.179 2374.73

Proposed 419.45 0.532 0.184 1757.66

Table 4. Software components fault-content and amount of testing-effort spent results

Models under Comparison

Reused Components

Newly Developed Components

Module 1 Module 2 Module 3 Module 4

a1 W1(t) a2 W2(t) a3 W3(t) a4 W4(t)

Yamada et al. (2000) Model 18.91 — 18.91 — 170.15 — 170.15 —

Proposed 51.59 8.94 32.30 2.03 158.13 13.65 177.43 30.21—the component is not part of the corresponding function

12 International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

by logistic function on the software reliability growth phenomenon.

The results of the proposed model are fairly encouraging when compared with other model developed under similar environment. The results can be viewed through the numeri-cal illustrations shown in Tables and Figures obtained after the estimation performed on real software reliability data sets. When applied to real software reliability data the proposed model provide a reasonable fit to the observed data and give insightful interpretations for the

resources consumption process during the test-ing phase. Following this, testing can be done more efficiently, by constantly monitoring the effort and the fault removal and then allocating optimal proportion of resources to the testing teams. We feel the proposed model can be of immense help in monitoring and controlling the resource utilization, measuring and controlling the progress of testing, efficiency of testing and reliability growth. The model can be modified according to the underlying testing environment. Therefore the model is being further modified

Figure 3. Non-cumulative fault removal curves

Figure 4. Cumulative fault removal curves

International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013 13

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

to incorporate the effects of imperfect debug-ging environment. Finally, the developed model provides a large scope for further extension and generalization.

ACKNOWLEDGMENT

The author acknowledges with gratitude the valuable comments and suggestions of the editor-in-chief and the referees, which helped in revising the manuscript.

REFERENCES

Buschmann, F., Henney, K., & Schmidt, D. C. (2007). Pattern-oriented software architecture: A pattern language for distributed computing (Vol. 4). Chichester, UK: John Wiley & Sons.

Cristescu, M., & Ciovica, L. (2010). Estimation of the reliability of distributed applications. Informatica Economica, 14(4), 19–29.

Goel, A. L., & Okumoto, K. (1979). Time dependent error detection rate model for software reliability and other performance measures. IEEE Transac-tions on Reliability, 28(3), 206–211. doi:10.1109/TR.1979.5220566.

Huang, C. Y., Kuo, S. Y., & Lyu, M. R. (2007). An assessment of testing-effort dependent soft-ware reliability growth models. IEEE Transac-tions on Reliability, 56(2), 198–211. doi:10.1109/TR.2007.895301.

Kapur, P. K., & Bardhan, A. K. (2002). Testing ef-fort control through software reliability modelling. International Journal of Modelling and Simulation, 22(1), 90–96.

Kapur, P. K., Bardhan, A. K., & Shatnawi, O. (2004a). Why software reliability growth modelling should define errors of different severity. Quality Control and Applied Statistics, 49(6), 699–702.

Kapur, P. K., Garg, R. B., & Kumar, S. (1999). Contributions to hardware and software reliability. Singapore: World Scientific. doi:10.1142/4011.

Kapur, P. K., Goswami, D. N., & Gupta, A. (2004b). A software reliability growth model with effort de-pendent learning function for distributed systems. International Journal of Reliability Quality and Safety Engineering, 11(4), 365–377. doi:10.1142/S0218539304001579.

Kapur, P. K., Gupta, A., Shatnawi, O., & Yadavalli, V. S. S. (2006). Testing-effort control problem us-ing flexible software reliability growth model with change-point. International Journal of Perform-ability Engineering, 2(3), 245–262.

Kapur, P. K., Pham, H., Anand, S., & Yadav, K. (2011). A unified approach for developing software reliability growth models in the presence of imperfect debugging and error generation. IEEE Transac-tions on Reliability, 60(1), 331–340. doi:10.1109/TR.2010.2103590.

Kapur, P. K., Singh, V. B., Anand, S., & Yada-valli, V. S. S. (2008). Software reliability growth model with change-point and effort using a power function of the testing time. International Journal of Production Research, 46(3), 771–787. doi:10.1080/00207540600926113.

Kuo, S. Y., Huang, C. Y., & Lyu, M. R. (2001). Framework for modelling software reliability using various estimating-effort and fault-detection rates. IEEE Transactions on Reliability, 50(3), 310–320. doi:10.1109/24.974129.

Lavinia, A., Dobre, C., Pop, F., & Cristea, V. (2011). A failure detection system for large scale distributed systems. International Journal of Distributed Sys-tems and Technologies, 2(3), 64–87. doi:10.4018/jdst.2011070105.

Lim, W. C. (1994). Effects of reuse on quality, productivity and economics. IEEE Transactions on Software, 11(5), 23–30. doi:10.1109/52.311048.

M. Lyu (Ed.). (1996). Handbook of software reli-ability engineering. New York, NY: McGraw-Hill.

Musa, J. D. (1999). Software reliability engineering. New York, N: McGraw-Hill.

Ohba, M. (1984). Software reliability analysis models. IBM Journal of Research and Development, 28(4), 428–443. doi:10.1147/rd.284.0428.

Ohtera, H., & Yamada, S. (1990). Optimal allocation and control problems for software-testing resource. IEEE Transactions on Reliability, 39(2), 171–176. doi:10.1109/24.55878.

Pham, H. (2000). Software reliability. Singapore: Springer-Verlag.

Raza, Z., & Vidyarthi, D. P. (2011). Reliability based scheduling model (RSM) for computational grids. International Journal of Distributed Sys-tems and Technologies, 2(2), 20–37. doi:10.4018/jdst.2011040102.

14 International Journal of Distributed Systems and Technologies, 4(2), 1-14, April-June 2013

Copyright © 2013, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

Shatnawi, O. (2007). Software reliability model for distributed systems. Al-Manarah Journal for Research and Studies, 13(6), 201–214.

Shatnawi, O. (2009a). Discrete time NHPP models for software reliability growth phenomenon. Inter-national Arab Journal of Information Technology, 6(2), 124–131.

Shatnawi, O. (2009b). Discrete time modelling in software reliability engineering: A unified approach. International Journal of Computer Systems Science and Engineering, 24(6), 391–398.

Wang, Z., Tang, K., & Yao, X. (2010). Multi-objective approaches to optimal testing resource allocation in modular software systems. IEEE Transac-tions on Reliability, 59(3), 563–575. doi:10.1109/TR.2010.2057310.

Xie, M. (1991). Software reliability modelling. Singapore: World Scientific Publishing.

Yamada, S., Ohba, M., & Osaki, S. (1983). S-shaped reliability growth modelling for software error detection. IEEE Transactions on Reliability, 32(5), 475–484. doi:10.1109/TR.1983.5221735.

Yamada, S., Ohtera, H., & Narihisa, H. (1985). Software reliability growth models with testing ef-fort. IEEE Transactions on Reliability, 35(1), 19–23. doi:10.1109/TR.1986.4335332.

Yamada, S., Osaki, S., & Narihisa, H. (1985). Software reliability growth models with two types of errors. Recherche Operationnelle / Operations Research (RAIRO), 19(1), 87-104.

Yamada, S., Tamura, Y., & Kimura, M. (2000). A software reliability growth model for a distrib-uted development environment. Electronics and Communications in Japan - Part 3, 83(12), 1–8. doi:10.1002/1520-6440(200012)83:12<1::AID-ECJC1>3.0.CO;2-4.

Zhao, X., Wang, T., Liu, E., & Clapworthy, G. J. (2010). Web services in distributed information systems: Availability, performance and composi-tion. International Journal of Distributed Sys-tems and Technologies, 1(1), 1–16. doi:10.4018/jdst.2010090801.

Omar Shatnawi is an Associate Professor in the Department of Computer Science of the Prince Hussein bin Abdullah Information Technology College at Al al-Bayt University, Jordan. He received his PhD in Computer Science and his M.Sc in Operational Research from University of Delhi in 2004 and 1999 respectively. His research interests are in software engineering, with an emphasis on improving software reliability and dependability.