chapter 2 literature review - inflibnetshodhganga.inflibnet.ac.in/bitstream/10603/45303/6/06_chapter...
TRANSCRIPT
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College, Gulaothi, Bulandshahar, CCS University, Meerut 28
Chapter 2
Literature Review
The Software Reliability Growth Model (SRGM) is the tool, which can be used to evaluate
the software quantitatively, develop test status, schedule status and monitor the changes in
reliability performance. There have been many Software Reliability models developed in
the last two decades. Most of these are based upon historical failure data collected during
the testing phase. These models have been utilized to evaluate the quality of the software
and for future reliability predictions. They have further been used in many management
decision-making problems that occur during the testing phase. But none of these models
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College, Gulaothi, Bulandshahar, CCS University, Meerut 29
can claim to be the best and hence there is a need for further research. In this section
different modelling approaches are briefly discussed.
2.1 Classification Scheme
The software reliability models are broadly classified into black box (single system) and
white box (multi component software) models. The black box models are classified
further into models based on
Inter Failure Times,
Failure Count and
Static Models.
Markov assumptions are also made for some models and they are overlapping with the
Failure Count and Inter Failure Time Modelling. All known models can be extended to be
a Bayesian model. If the model is Bayesian then we are estimating the model parameters
using Bayesian techniques. White box models are those that model modular software
systems considering the architecture of the system.
Popstajanova and Trivedi (Popstajanova and Trivedi 2001) presented a classification of
software reliability models that classified multi component software systems as stated in
Figure 2.1.
The white box models are broadly classified into State Based, Path Based and Additive
Models. The State Based Models are further classified into Continuous Time Markov
Chains (CTMC), Discrete Time Markov Chain (DTMC) and semi-Markov models.
All these Markov models fall into either absorbing or irreducible class of Markov chains.
Absorbing type represents terminating software and continuously operating system is
represented by irreducible chain.
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College, Gulaothi, Bulandshahar, CCS University, Meerut 30
Figure 2.1 : Classification Scheme
Software Reliability Models
Black Box Models
(Single system)
White Box Models(Multi Components)
Type I State Based Models
Failure rate models
Type IIBayesianModels
Static Models
Fault Seeding
Input Domain Models
Markov Reliability Models
DTMC (Discrete)
CTMC (Continuous)
Semi Markov
Absorbino
Irreducible
Composite
Hierarchical
Path BasedModels
Additive Models
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College, Gulaothi, Bulandshahar, CCS University, Meerut 31
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 32
Bayesian models are extensions of Software Reliability Growth Models. In Bayesian models the
stochastic model reflects the system accurately at that instant based on the latest failure data. In
Bayesian models the distributions representing the failure behaviour is represented by a prior and
posterior distribution. The prior distribution reflects the expert knowledge available on the
specific failure process, which is incorporated in the stochastic model. Bayesian models have
wide acceptance due to these reasons in the software industry. In the following sections we
describe each class of software reliability models with a representative reliability model.
Single System Software
This model considers the whole software as a single monolithic system and the structure of the
model is not considered in the process of reliability estimation of the software system. The
reliability is solely estimated based on the failure history. Popular reliability estimation models
include the Goel-Okumoto (Goel and Okumoto 1979) model, and the Jelinski-Moranda model
(Jelinski and Moranda 1972). Thus, for example, these models do not account for the structure of
the software. Single system software failure behaviour is usually modelled using Software
Reliability Growth Model and classified as times between failure and failure count models.
Failure Rate Models
Inter failure times are the main modelling parameter for these models. Generally it is expected
that the time for next failure increase as more bugs are detected and corrected. This may not be
always correct as inter failure times are random variables and subjected to statistical fluctuations.
The failure rate is a modelling tool for type I models. The reliability function is a non-decreasing
function of mission time. This demonstrates an increase in the software credibility.
Markovian Models
When a markov process represents the failure process, the resultant model is called Markovian
model. The software can attain many states at any particular time with respect to number of
faults remaining or number of faults already removed. The transition between states depends on
the current state of the software and the transition probability. The memory less property of the
Markovian process implies that the time between failures follows an exponential distribution.
Numerous attempts have been made to develop Markovian models; especially in earlier days due
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 33
to similar theory in hardware reliability was already well developed. One of the popular
reliability models developed by Jelinski and Moranda (Jelinski and Moranda 1972) is a Markov
process model. Littlewood (Littlewood 1973, Littlewood 1979, Littlewood 1987) proposed a
model based on semi-Markovian process to describe the failure phenomenon of software with
module structure. Cheung (Cheung 1980) has also proposed a Markovian model to describe
module-structured software. Kremer (Kremer 1983) proposed birth-death model, which
incorporates the probabilities of fault removal and introduction. Goel (Goel 1985) modified
Jelinski-Moranda model by introducing the concept of imperfect debugging. Kapur et al. (Kapur,
Garg and Kumar 1999) proposed a model incorporating imperfect debugging and fault
introduction with upper bound on the number of faults.
Fault Counting Models
This category includes the models, which describe the failure phenomenon by stochastic
processes like Homogeneous Poisson Process (HPP), Non-Homogeneous Poisson Process
(NHPP), and Compound Poisson Process (CPP) etc. The majority of these failure count models
are based upon the NHPP. Schneidewind (Schneidewind 1975) proposed a fault detection model
based on NHPP. Different NHPP models are distinguished by their unique mean value functions.
Yamada et al. (Yamada, Ohba and Osaki 1983) proposed the delayed S-Shaped model. Ohba
(Ohba 1984) proposed the inflection S-Shaped model. Musa et al. (Musa, Iannino and Okumoto
1987) have proposed the basic execution time model and Log-Poisson model. Goel (Goel 1985)
modified Musa’s original model by introducing the test quality parameter. Yamada et al.
(Yamada, Osaki and Narithisa 1985) also proposed a discrete time model. The effect of testing
effort on failure process was taken into consideration by Yamada et al. (Yamada, Ohtera and
Narithisa 1986). Kapur et. al (Kapur, Garg and Kumar 1999) modified Goel-Okumoto (Goel and
Okumoto 1979) model by introducing the concept of imperfect debugging. Kareer et al. (Kareer
1990) proposed a model with two types of faults where each fault type is modelled as by an S-
Shaped curve model. Xie and Zhao (Xie and Zhao 1992) have illustrated how the Schneidewind
model can be modified to result in many of the above SRGMs. Zeephongsekul et al.
(Zeephongsekul P, Xia G and Kumar S 1993) proposed a model describing the case when a
primary fault introduces secondary faults. Attempts as listed above for new models were made
with the primary intention of getting flexible models that could describe a range of failure count
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 34
curves or reliability growth curves like exponential curves and highly S-Shaped curves. Models
with such property are termed as flexible SRGMs (Kapur, Garg and Kumar 1999). Recently
efforts have been directed towards development of general SRGMs (Kapur, Garg and Kumar
1999). General SRGMs are flexible models and many of the above models can be derived from
them. In this thesis new NHPP based SRGMs are proposed and the theory and methodologies
developed in the rest of the thesis are based upon NHPP models. Therefore later in this chapter
we briefly discuss the Non-Homogeneous Poisson Process and some of the above SRGMs.
Recently reliability modelling for distributed development environment has caught the attention
of many researchers. Large software systems have modular design. A system is said to be
modular when each activity of the system is performed by exactly one component, and when the
inputs and outputs of each component are well defined (Lyu 1996). Often different development
teams develop such components of software separately. With availability of communication
networks at cheaper rates some software components are developed at separate geographic
locations also. Software developed under this distributed development environment has proved
to be economical. Many a time components from other software projects are also reused.
Therefore SRGMs for software developed under distributed environment needs to have different
approach. But very few attempts have been made in this regard (Yamada, Tamura and Kimura
2000). Most of the models proposed in the thesis have been developed under distributed
development environment, which assumes that the software system consists of a finite number of
reused and newly developed components.
Models based on Bayesian Analysis
In the previous two categories the unknown parameters of the models are estimated either by the
least squares method or by the maximum likelihood method (later in this chapter both these
methods are briefly discussed). But in this category of models, the Bayesian analysis technique is
used to estimate the unknown parameters of the models. This technique facilitates the use of
information obtained by developing similar software projects. Based on this information the
parameter of given model are assumed to follow some distribution (known as priori distribution).
Given the software test data and based on a priori distribution, a posterior distribution can be
obtained which in turn describes the failure phenomenon. Littlewood and Verral proposed the
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 35
first software reliability model based on Bayesian Analysis (Xie 1993). Singpurwalla
(Singpurwalla 1995, Singpurwalla 1999) have proposed a number of Bayesian software
reliability models for different testing environments.
The following four categories of Software Reliability models do not make any dynamic
assumptions of the failure process. These are defined very briefly here.
Static Models
These models use statistical techniques to evaluate software reliability and can be evaluated only
if complete failure data is available. These models were developed in the initial stage of
reliability model evolution and are now seldom used, as it cannot incorporate the structure of the
software.
The Input Domain and Fault Seeding Models
In fault seeding models, a known number of bugs are seeded (planted) in the program. The
software is then tested for faults. The bugs detected would have a combination of inherent faults
and seeded faults. The number of inherent faults in the software is calculated based on the
inherent and seeded faults detected using maximum likelihood estimation and combinatory. The
drawback of this approach is that the seeded faults and the inherent faults must have the same
detection probability. This is difficult to achieve. The basic approach in input domain based
model is to generate a set of test cases from an input distribution. The reliability measure is
calculated from the number of failures observed during symbolic or physical execution of
sampled test cases. The test cases selected from the representative input space are executed
recording the results. The probability of the success can be evaluated using statistical techniques.
It is generally difficult to estimate the input distribution (operational profile); generally the input
distribution is obtained based on the different paths that exist in the software.
Software Metrics Models
The models in this class relate the fault content in the software to some features of the software
program such as program length, complexity, volume etc. These models are empirically built and
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 36
the result obtained by a model is dependent on the software development process environment,
which may not be the same in the other projects.
Models other than those falling under above categories exist in the literature and more are being
proposed. A number of review papers have been written, to name among others Littlewood
(Littlewood 1973), Goel (Goel 1985), Xie (Xie 1991), Kapur et al. (Kapur, Garg and Kumar
1999). In the following section we briefly describe some Software Reliability Growth models
based upon NHPP assumptions. Estimation procedures and Criteria for comparison among
Software Reliability Growth Models are also discussed.
Multi Component Models
White Box model or Multi component system considers the Software Architecture of the system,
its interactions with the different modules. White box models are used to model component
based software. Such models seek to explicitly incorporate the testing method used during the
testing phase, as well as the structure of the software being tested. Modular systems have a
general framework in which the system reliability is calculated. All modular systems are grouped
as State Based, Path Based and Additive models.
State Based Models
Architecture based models assume that components fail independently and that a component
failure will ultimately lead to a system failure. Unlike hardware reliability every component is
always in use, software components need a utilization factor in State Based Models. State Based
Models are generally modelled using Markov models like CTMC, DTMC or semi Markov
models. System reliability estimates are obtained using both architecture and failure model. This
is achieved using two methods, Composite and Hierarchical solution approach.
Composite-Hierarchical Approach
The State Based Models are further classified into composite and hierarchical based on the
solution approach to obtain the reliability of the system. Composite method combines the
architecture model with the failure model and then solved for reliability prediction. If the
architecture model is first solved first and then superimposed on the failure behaviour on the
architecture model solution to predict reliability.
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 37
Path Based Models
Similar to State Based Models, the Path Based Models consider software architecture with
components and interfaces. Initially the different paths in system are obtained
either experimentally or algorithmically. Path reliability is the product of all component
reliabilities along the path. The system reliability is average of all the path reliabilities. State
based models analytically account for the infinite loops in a path but path-based models
terminate the loop to one or to an average execution time of the path. Shooman Model considers
reliability of modular software introducing the path based approach by using the frequencies
with which different paths are run. Krishnamurthy and Mathur developed a method to combine
architecture and failure process by estimating the path reliabilities based on the sequence of
components executed for a single test run and the average over all test runs to obtain the system
reliability. State Based Models are an important category of models for modular systems.
Additive Models
Additive models consider software testing phase and each component reliability is modeled by
NHPP. This implies system failure rate is also NHPP with cumulative number of failures and
failure intensity functions that are the sums of corresponding function of each component.
Additive model do not consider architecture of the software. Xie & Wohlin (Xie & Wohlin 1995)
developed an additive based architecture model. Semi Markov and Markov regenerative models
attempt to relax the Markovian assumption of exponential failure and repair times description, in
a restrictive manner. They are also exposed to the state space explosion problem. Discrete-event
simulation on the other hand offers an attractive alternative to analytical models as it can capture
a detailed system structure and facilitate the study of influence of various factors such as
reliability growth, various repair policies. All software development teams have to answer an
obvious question of when to stop testing and release the software, whether the software is single
component or multi-component.
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 38
2.2 NHPP based Software Reliability Growth Modeling
Stochastic modeling has been to develop models to represent the real system and analyze their
operation since years. There are two main types of stochastic process: continuous and discrete.
Among the class of discrete process, counting process is used in reliability engineering widely to
describe the occurrence of an event of time (e.g. failure, repair, etc.). Poisson process is used
most widely to describe a counting process as in reliability engineering. NHPP has been used
successfully in hardware reliability analysis to describe the reliability growth and deteriorating
trends. Following the trends in hardware reliability analysis many researchers proposed and
validated several NHPP based SRGM. SRGM describe the failure occurrence and\or failure
removal phenomenon with respect to time (CPU time, calendar time or execution time or test
cases as unit of time) and\or resources spent on testing and debugging during testing and
operational phases of the software development.
NHPP based SRGM are broadly classified into two categories first – continuous time models,
which uses time (CPU time, calendar time or execution time) as a unit of fault detection period
and second – Discrete time models, which adopt the number of test occasions/cases as a unit of
fault detection period. Models can also be categorized as concave and S-shaped depending upon
the shape of the failure curve described by them. Concave models describe an exponential failure
curve while second category of models describes an S-shaped failure curve [Kapur et al, (1999);
Pham, (2006)]. The two types of failure growth curves are shown in figure 2.2 and 2.3. The
most important property of these models is that they have the same asymptotic behavior in the
sense that the fault detection rate decreases as the number of detected defects increases and
approaches a finite value asymptotically. The S-shaped curve describes the early testing process
to be less efficient as compared to the later testing i.e. it depicts the learning phenomenon
observed during testing and debugging process.
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 39
Figure 2.2 Exponential Failure Curve Figure 2.3 S-Shaped Failure Curve
During the last three decades several researchers devoted their research interest to NHPP based
software reliability modeling and contributed significantly in understanding the testing and
debugging process and developing quality software. Schneidewind, (1975) made the preliminary
attempt in NHPP based software reliability modeling. He assumed exponentially decaying failure
intensity and rate of fault correction proportional to the number of faults to be corrected. Goel
and Okumoto, (1979) presented a reliability model (GO model); assuming hazard rate is
proportional to the remaining number of faults in the software. This research paper was a
pioneering attempt in the field of software reliability growth modeling and paved the way for
research on NHPP based software reliability modeling. The model describes the failure
occurrence phenomenon by an exponential curve. The research following GO model was mainly
modifying the existing research in the way of incorporating the various aspects of the real testing
environment and strategy.
In the upcoming sections we briefly discuss some of the most widely known/preliminary
attempts in some of the above modeling categories.
2.3 General Description of a Non-Homogeneous Poisson Process
2.3.1 NHPP in Continuous Time Space
A counting process ( ( ), 0)N t t is said to be an NHPP with mean value function m(t), if it
satisfies the following conditions:
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 40
1. There are no failures experienced at t=0, that is, 0)0(N .
2. The counting process has independent increments, i.e. for any finite collection of times
t1<t2<�<tk, the k random variables N(t1), N(t2)-N(t1),…, N(tk)-N(tk-1) are independent.
3. Pr(exactly one failure in ( , )) ( ) ( )t t t t o t
4. Pr(two or more failures in ( , )) ( )t t t o t
Where ( )t is intensity function of ( )N t . If we let 0
( ) ( )t
m t x dx them m(t) is a non-
decreasing, bounded function representing the mean of number of faults removed in the time
interval (0, ]t [Kapur et al, (1999)]. It can be shown that
- ( )( )Pr[ ( ) ]
!
k m tm t eN t k
k, 0,1, 2,....n
i.e. ( )N t has a Poisson distribution with expected value [ ( )] ( )E N t m t for 0t .
and the reliability of the software in the time interval of length x is given as
- ( ) ( )( | ) m t x m tR x t e
In case we have n, the number of test cases as the unit of time then t is replaced by n and t is
replaced by 1, and describes NHPP based SRGM in discrete time space.
2.3.2 NHPP in Discrete Time Space
A discrete counting process[ ( ), 0]N n n , ( 0,1, 2,...)n is said to be an NHPP with mean value
function ( )m n , if it satisfies the following two conditions
1. No failures are experienced at n=0, that is, (0) 0N .
2. The counting process has independent increments, implies the number of failures
experienced during thth , 1n n test cases is independent of the history. The state ( 1)m n
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 41
of the process depends only on the present state ( )m n and is independent of its past state ( )m x ,
for x n .
For any two test cases in and jn where (0 )i jn n , we have
( ) ( )Pr{ ( ) ( ) } exp ( ) ( )
!
xj i
j i j im n m n
N n N n x m n m nx
, x = 0,1,2,
The mean value function )(nm which is a non-decreasing in n represents the expected
cumulative number of faults detected by n test cases. Then the NHPP model is formulated as
( )( )Pr{ ( ) } exp
!
xm nm n
N n xx
Let ( )N n denote the number of faults remaining in the system after execution of the thn test run.
Then we have
( ) ( ) ( )N n N N n
where, ( )N represents the total initial fault content of the software.
The expected value of ( )N n is given by
( ) ( ) ( )E n m m n
where, ( )m represents the expected number of faults to be eventually detected.
Suppose that dn faults have been detected by the execution of n test cases. The conditional
distribution of ( )N n , given that ( ) dN n n , is given by
( )( )Pr{ ( ) | ( ) } exp
!
yE n
dE n
N n y N n ny
and the probability of no faults detected between the thn and th( )n h test cases, given that dn
faults have been detected by n test cases i.e. software reliability, is given by
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 42
( / ) exp ( ) ( )R h n m n h m n ,
2.4 NHPP based SRGM: Literature Survey
A large number of SRGMs have been developed in the literature to describe the failure and fault
removal phenomenon of software in continuous and discrete time space. In this section we
introduce briefly some of the existing SRGM in the literature
Notations
m(t) : expected number of failure/removal by time t, m(0) = 0.
mf(t) : expected number of failure by time t, mf(0) = 0.
mr(t) : expected number of removals by time t, mr(0)=0.
mi(t) : expected number of failure due to independent faults in time t, mi(0) = 0.
md(t) : expected number of failure due to dependent faults by time t, md(0)=0.
a : number of fault in the software at the time of start of software testing.
a(t) : expected initial fault content at time t, a>0.
p : probability of perfect debugging of a fault, 0<p<1.
b(t) : time dependent rate of fault removal per remaining faults.
, c : constants
2.4.1 Modeling Under Perfect Debugging Environment
The earlier attempts in NHPP based reliability growth modeling assumed a perfect debugging
environment. Which means whenever an attempt is made to remove a detected fault it is removed
perfectly and no new faults are generated. Various models have been proposed in this category.
The earliest model was due to Goel and Okumoto, (1979).
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 43
The Exponential SRGM [Goel and Okumoto, (1979)]
The model is based on the following assumptions
1. Failure observation phenomenon is modeled by NHPP.
2. Software is subject to failures during execution caused by faults remaining in the software.
3. Each time a failure is observed, an immediate effort takes place to isolate and remove the
cause of the failure.
4. All faults in the software are mutually independent.
Under the assumptions 1- 4 the differential equation describing the GO model is
( ) [ ( )]dm t dt b a m t (2.1)
solving (2.1) with the initial condition m(0)=0 we get
( ) (1 )btm t a e (2.2)
The model is known as exponential NHPP model as it describes an exponential failure curve.
GO model has been applied to a variety of testing environment in practice. Following, the
general assumptions of GO model other early exponential SRGM are proposed by Obha, (1984)
and Yamada and Osaki, (1985). Obha assumed that the software consists of a number of
independent modules whereas Yamada and Osaki assumed there are two types of faults in the
software. Both these models describe the failure phenomenon for each module\fault type by GO
model with different parameters and the mean value function is the sum of mean value function
for each module\fault type. Musa, (1975) and Musa and Okumoto (1984) assumed failure
intensity to decay exponentially in the basic and Logarithmic Poisson execution time models
respectively. Yamada et al, 1983 refined the GO model describing testing as a two-stage process
namely fault detection and removal. The model describes an S-shaped failure curve. SRGM
proposed by Obha, (1984a); Bittanti et al, (1988) and Kapur and Garg, (1992) are similar to
Yamada et al, (1983) but are developed under different set of assumptions. These models can
describe both exponential and S-shaped growth curves depending on the parameter values and
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 44
therefore are termed as flexible models or S-shaped models. Some of these models are discussed
here.
Yamada Delayed S-shaped Model [Yamada et al, 1983]
This model defines the testing as a two-stage process - failure observation and the corresponding
fault removal phenomenon.
( ) ( ( ))f fdm t dt b a m t (2.3)
( ) ( ( ) ( ))r f rdm t dt b m t m t (2.4)
solution of above differential equations under initial condition mf(0)= mr(0)= 0, is
( ) (1 )btfm t a e (2.5)
( ) (1 (1 ) )btrm t a bt e (2.6)
Model can be derived alternatively in single stage if we assume 2( ) (1 )b t bt as the fault detection
rate [Kapur et al, (1999, 2011)].
SRGM for Error Removal Phenomenon [Kapur and Garg, (1992)]
This model is based on the assumption that the debugging team can also remove some additional
faults while removing some faults that have caused a failure without these faults causing any
failure. The faults identified on a failure are termed as independent faults while the faults
removed additionally are termed as dependent faults.
( )( ) ( ( )) ( )r
r r rm td m t p a m t q a m t
dt a (2.7)
The mean value function obtained under the initial condition mr (0) = 0 is
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 45
( )
( )1
( )1
p q tr p q t
em t aq p e
(2.8)
If c=0 the model reduces to GO model and failure phenomenon is same as removal else the
failure phenomenon is given as
( )0( ) ( ) ln
tf r p q t
ap p qm t p a m x dxq p qe
(2.9)
The model can be derived alternatively if we assume logistic learning fault detection rate given
by ( ) 1 btb t b e with b=(p+q) and =(q/p).
2.4.2 Reliability Growth Modeling with Testing Effort
Most of these earlier SRGM use calendar (execution) time as the unit of fault detection/removal
period and either assume that the testing effort consumption (TEC) rate is constant, or do not
explicitly consider the testing efforts (TE) and its effectiveness. The achieved reliability during
testing phase is highly related to the amount of development resources (test efforts) spent on
detecting and correcting latent software faults. The TE are measured by the manpower spent
during the testing phase, the number of CPU hours, the number of executed test cases, and so on.
A testing effort function describes the distribution of TE (CPU time, manpower, etc) during the
testing period. Putnam et al (1978), Yamada et al (1983,1985,1991,1993), Kuo et al (2001),
Kapur et al (2004b, 2004c), Bokhari and Ahmad (2006) and Huang et al (2002,2007) proposed
SRGM describing the relationship among the testing time, TEC and the number of software
faults detected. Most existing SRGM belong to exponential-type models. Haung et al (2007)
proposed S-shaped TE dependent SRGM based on Yamada delayed S-shaped model. In the
literature the time-dependent behavior of test-effort function is described by Weibull, Rayleigh,
exponential, Gamma, logistic, log logistic curves etc.
The testing effort (resources) that govern the pace of testing for almost all the software projects
are (Musa, Iannino and Okumoto 1987):
1. Manpower that includes
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 46
i) Failure identification personnel
ii) Failure correction personnel.
2. Computer time
These resources very closely interact during the system test phase. The function of the failure
identification personnel is to run test cases and compare the test results against program
requirement to establish the failures that have occurred. The failure correction personnel are the
debuggers or developers available to repair the software. The Computer facility represents the
computer(s) necessary for the failure identification personnel and failure correction personnel to
do their tasks. Computer time is the measure that is used for allocating computer resources.
SRGMs have been proposed in the Software Reliability Engineering literature under sets of
assumptions and for different testing environments. But most of them do not consider the
consumption pattern of resources such as computer time and manpower during testing. In this
thesis some new SRGMs incorporating testing effort have been proposed.
As the time increases, testing effort also increases. If testing time becomes quite large, testing
effort also becomes quite large. The testing effort rate is proportional to the testing resources
available i.e.,
)()()( tWtv
dttdW
(2.10)
where is the total testing resource consumption and )(tv is the time dependent rate at which
testing resources are consumed, applied to remaining available resources. Solving equation
(2.10) under the initial condition 0)0(W we get:
0
( )
( ) 1
t
v x d x
W t e (2.11)
Though various types of testing effort function like Exponentials, Rayleigh, Weibull and Logistic
have been studied in the literature. In this thesis, we have used weibull function to model the
testing resources modeling. The detailed description of these effort functions is given in chapter
6.
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 47
Model using Testing Effort Function (Yamada et al 1983)
Under the assumptions described above and using equation (2.11), the fault removal process can
be described by the following differential equation:
tmatbtw
dttmd
)(
)(
(2.12)
Using the different forms of b(t) several SRGMs can be formulated
Let the fault detection rate function is defined as:
b t b (2.13)
Here w(t) is the current testing effort consumption at time t.
.1 b W tm W t a e (2.14)
The above model is known as GO Model with respect to Testing Effort Function Yamada et al
(1983)
2.5 Multi Up-gradations of Software
The present software development environment is very competitive and advanced. Many
independent and well established developing organizations are competing in the market with
similar products and capabilities to attain the maximum market share and brand value. As such
software delivered with full functionalities and very high reliability built over a period of time
may turn out to be unsuccessful due to technological obsolescence. Therefore now a day’s the
software are rather developed in multiple releases where the latest releases might be developed
by improving the existing functionality and revisions, increasing the functionality, a combination
of both or improving the quality of the software in terms of reliability etc. For example we can
see the various software in the market named as Windows 98, Windows 2000, Windows ME,
Windows XP, Windows Vista, Windows 7 etc. For another illustration consider a development
firm developing antivirus software. Such a firm can begin with releasing the product that detects
and remove viruses and spywares from the computer system. In their second release they can
provide the feature of protecting the system from virus infected emails. Next, they can add the
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 48
trait of blocking spyware automatically for the next release. Finally, the fourth release can
provide the root kit protection along with removing hidden threats in the operating system.
This step by step release (base software with features enhancement) is advantageous for the
developing firms in various contexts. Firstly, if a firm implements the complete characteristic
capabilities in first release, than that would delay the product release in the market in the desired
time window. Secondly, launching of new software product may bring the developing firm in
limelight, but the stream of subsequent product releases is the source of their bread and butter.
Moreover releasing different versions of the product lengthen the market life of product, protect
competitive advantages and sustain crucial maintenance revenue streams.
Software products aren’t static and each release has a limited existence. As soon as a software
product reaches the market, a variety of factors begin creating demand for changes (Figure 2.4 ).
Defects require repairs. Competitors offer software with added features and functions. Evolving
technology requires upgrades to support new hardware and updated versions of operating
software. Sales demands new features to win over prospects. Customers want new functionality
to justify maintenance expenditures. These demands accumulate over time, eventually reaching
the point when the software product must be upgraded with a new version to remain viable in the
market. As soon as the new version is released, the cycle begins again.
For software developing organizations it is not an easy task to design software in isolation.
Developing reliable software is one of the most difficult problems faced by them. Timely
development, resource limitations, and unrealistic requirements can all negatively impact
software reliability. Moreover, there is some interdependence between their developments. The
interdependence between their developments exists in many ways, which also affects their
reliability. A new release (an upgraded version of the base software) may come into existence
even during its development, at the time of release or during its operation. The code and other
documents related to a release may be some modification of the existing code and documents
and/or addition of new modules and related modification in the documents. The dependence of
the development process of successive releases necessitates considering this dependence in the
reliability growth analysis.
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 49
Figure 2.4: Necessity for Up-gradations
Not much work has been done in the modeling framework for multi releases failure process. In
the earlier work Kapur et al (2007) proposed initial development in this field by proposing
multiple releases of a software in operational phase for product and project type software. Later
Kapur et al (2010) also described the phenomenon specially for testing phase by reformulating
modeling methodology that shows the dependence of operational phase of the previous release
with testing phase of the current release. In this thesis we have formulated SRGMs applicable to
describe the failure\removal phenomenon of multiple releases of software in testing phase and
also considered its dependency with operational phase wherein the expected number of failure
and consequently the faults removed in testing phase is modeled by the joint effect of
failure\removal phenomenon of successive releases.
2.6 Parameter Estimation
The success of mathematical modeling approach to reliability evaluation depends heavily upon
quality of failure data collected. The parameters of the SRGMs are estimated based upon these
data. Hence efforts should be made to make the data collection more explicit and scientific.
Usually data is collected in one of the following two ways. In the first case the times between
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 50
successive failures are recorded. Though this type of data collection is more desirable, it may not
be simple. Complication can arise in measuring the testing effort for each fault and it may not be
very convenient to note the time at each failure report. The other easier and commonly collected
data type is known as the grouped data. Here testing intervals are specified and number of
failures experienced during each such interval is noted. The proposed models presented in the
thesis are non-linear and presents extra problems in estimating the parameters.
Technically, it is more difficult to find the solution for non-linear models using Least Square
method and requires numerical algorithms to solve it. Statistical software packages such as SPSS
helps to overcome this problem. SPSS is a Statistical Package for Social Sciences. It is a
comprehensive and flexible statistical analysis and data management system. SPSS can take data
from almost any type of file and use them to generate tabulated reports, charts, and plots of
distributions and trends, descriptive statistics, and conduct complex statistical analysis.
SPSS Regression Models enables the user to apply more sophisticated models to the data using
its wide range of nonlinear regression models. For the estimation of the parameters of the
proposed models, Method of Least Square (Non Linear Regression method) has been used. Non
Linear Regression is a method of finding a nonlinear model of the relationship between the
dependent variable and a set of independent variables. Unlike traditional linear regression, which
is restricted to estimating linear models, nonlinear regression can estimate models with arbitrary
relationships between independent and dependent variables.
2.6.1 METHOD OF LEAST SQUARES
In this method the square of the difference between observed response and value predicted by the
model is minimized. If the expected value of the response variable is given by )(ˆ tm (can be a
mean value function of an SRGM), then the least square estimators of the parameters of the
model may be obtained from n pairs of sample values (t1, y1), (t2, y2), …, (tn, yn) by minimizing J
given by
n
ii tmyJ
1
2)(ˆ (2.15)
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 51
ti and yi observed values of explanatory and dependent variables respectively. For small and
medium size samples least square estimation is preferred (Musa, Iannino and Okumoto 1987).
The method has also been applied for estimating parameter of testing effort curves.
2.6.2 MAXIMUM LIKELIHOOD ESTIMATION
Maximum Likelihood Estimation (MLE) method has been extensively adopted for estimation of
parameters of SRGMs based upon NHPP (Musa, Iannino and Okumoto 1987). We briefly
discuss below the MLE procedure for two types of software failure data discussed above.
For the first type of data, suppose that estimation is to be performed at a specified time tk, not
necessarily corresponding to a failure, and with total of mk failures being experienced at time
kmttt ,...,, 21 . Then the likelihood function for the NHPP (Musa, Iannino and Okumoto 1987)
discussed above is:
ktdxxk
ii etL 0
)(
1
)( (2.16)
The MLE of the Parameters can be obtained by maximizing Likelihood function or its Log
likelihood function (log L).
If the software failure data is grouped into k points ),( ii yt ; i = 1,2,…,k, where yi is the
cumulative number of failure reports at time ti. Then the Likelihood function L is given as
follows:
k
i
tmtm
ii
yyii ii
ii
eyytmtmL
1
)()(
1
1 1
1
)!(
)()( (2.17)
Taking natural logarithm of (2.17) we get the loglikelihood function
k
iii
k
ikiiii yytmtmtmyyLogL
11
111 ])!ln[()()]()(ln[)( (2.18)
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 52
The MLE of the parameters of SRGM can be obtained by maximizing (2.18) with respect to the
model parameters.
Likelihood functions for NHPP models are defined above as most of the stochastic models
discussed in this thesis are based upon NHPP assumptions. Both the estimation procedures can
also be applied to other stochastic processes. Maximum likelihood estimators possess many
desirable properties such as consistency, efficiency, asymptotic normality and the invariance
property (Musa, Iannino and Okumoto 1987). Hence it is the most preferred estimation
procedure for relatively large sample size.
2.7 Comparison Criteria for SRGMs
The performance of SRGMs are judged by their ability to fit the past software fault data
(goodness of fit) and to predict satisfactorily the future behaviour of the software fault removal
process (predictive validity). Kapur, Garg and Kumar (1999), Musa (1987, 1999) have suggested
the following attributes for choosing an SRGM.
Capability: The model should possess the ability to estimate with satisfactory accuracy metrics
needed by the software managers.
Quality of assumptions: The assumptions should be plausible and must depict the testing
environment.
Applicability: A model can be adjudged as the better one if it can be applied across software
products of different sizes, structures, platforms and functionalities.
Simplicity: The data required for an ideal SRGM should be simple and inexpensive to collect.
The parameter estimation should not be too complex and is easy to understand and apply even
for persons without extensive mathematical background.
Other than the above qualitative aspects the following indices help in comparing SRGMs.
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 53
2.7.1 GOODNESS OF FIT CRITERIA
The term goodness of fit is used in two different contexts. In one context, it denotes the question
if a sample of data came from a population with a specific distribution. In another context, it
denotes the question of “How good does a mathematical model (for example a linear regression
model) fit to the data”?
1. The Mean Square Fitting Error (MSE): The model under comparison is used to
simulate the fault data, the difference between the expected values, )(ˆ itm and the observed data
yi is measured by MSE as follows.
k
i
iik
ytmMSE1
2))(ˆ( (2.19)
where k is the number of observations. The lower MSE indicates less fitting error, thus better
goodness of fit (Kapur, Garg and Kumar 1999).
2. Coefficient of Multiple Determination (R2) This Goodness-of-fit measure can be used
to investigate whether a significant trend exists in the observed failure intensity. We define this
coefficient as the ratio of the Sum of Squares (SS) resulting from the trend model to that from a
constant model subtracted from 1, that is
SScorrectedSSresidualR 12 (2.20)
R2 measures the percentage of the total variation about the mean accounted for by the fitted
curve. It ranges in value from 0 to 1. Small values indicate that the model does not fit the data
well. The larger, the better the model explains the variation in the data (Kapur, Garg and Kumar
1999).
3. Prediction Error (PE): The difference between the observation and prediction of
number of failures at any instant of time i is known as PEi. Lower the value of Prediction Error
better is the goodness of fit (Pillai and Nair 1997).
4. Bias: The average of PEs is known as bias. Lower the value of Bias better is the
goodness of fit (Pillai and Nair 1997).
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 54
5. Variation: The standard deviation of PE is known as variation.
2
11 BiasPENVariation i (2.21)
Lower the value of Variation better is the goodness of fit (Pillai and Nair 1997).
6. Root Mean Square Prediction Error: It is a measure of closeness with which a model
predicts the observation.
22 VariationBiasRMSPE (2.22)
Lower the value of Root Mean Square Prediction Error better is the goodness of fit (Pillai and
Nair 1997)
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 55
References
ANSI/IEEE, Standard Glossary of Software Engineering Terminology, STD-729-1991.
Bittanti S., Bolzern P., Pedrotti E., Pozzi N. and Scattolini R., “A flexible modelling approach for software reliability growth”, Goos, G., and Harmanis, J., (Ed.), Software Reliability Modelling and Identification, Springer Verlag, Berlin, pp. 101-140, 1988.
Bittanti S. (Editor) “Software Reliability Modelling and Identification” Lecture notes in Computer Science Berlin: Springer-Verlag, 1987.
Brooks W.D. and Motley R.W., “Analysis of Discrete Software Reliability Models-Technical Report (RADC-TR-80-84)”, Rome Air Development Center, New York, 1980.
Bardhan AK. “Modeling in Software Reliability and its Interdisciplinary Nature”. Ph.D. Thesis, University of Delhi, Delhi, 2002.
Chen Y., Singpurwalla N. D., Unification of software reliability models by self-exciting point processes, Adv. Appl. Probab., 29, 337–352, 1997.
Daisuke Satoh, Yamada Shigeru, “discrete equation and software reliability growth models” IEEE, pp. 176-184, 2004.
Gupta Anu “Some Contributions to Flexible Software Reliability Growth Modelling –Ph.D. Thesis” University of Delhi, Delhi, 2007.
Garg RB. “Reliability Modelling In Software and Computing Systems-Ph.D. Thesis” University of Delhi, Delhi, 1990.
Goel A.L. and K. Okumoto, “Time dependent error detection rate model for software reliability and other performance measures”, IEEE Transactions on Reliability, Vol. R-28 No. 3, pp. 206-211, 1979.
Gokhale S.S., Lyu M.R. and Trivedi K.S., “Analysis of software fault removal policies using a non-homogenous continuous time Markov chain”, Software Quality Journal, vol. 12, pp.211-230, 2004.
Goswami, DN. “Software Reliability Growth Modelling And Its Impact On Testing Resources- Ph.D. Thesis”, Jiwaji University, Gwalior, 2004.
Huang C. Y., Lyu M. R. and Kuo S. Y., “A unified scheme of some non-homogeneous Poisson process models for software reliability estimation”, IEEE Trans. on Software Eng., 29, pp. 261–269, 2003.
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 56
Huang Yu Chin “Performance Analysis of Software Reliability Growth Models with Testing Effort and Change-Point “, Journal of Systems and Software Vol. 76, Issue 2, pp 181-194, 2005.
Innoue S. and Yamada S. “Generalised Discrete Software Reliabilty Modelling with effect of Program Size”, IEEE transactions Syst. Man Cybern. A, Syst. Humans, vol. 37,No.2, pp. 170-179,2007.
Inoue S. and Yamada S., “An infinite server queuing model considering time distribution of fault isolation process for software reliability assessment”, (in Jananese) IEICE Technical Report [Reliability], Vol. 102 No. 454, pp. 119-124, 2002.
Inoue, S. and S. Yamada, Discrete Software Reliability Assessment with Discretized NHPP Models, Computer & Mathematics with Applications, Vol. 51, No. 2, pp.161-170, 2006.
Kapur P. K., Garg R.B. and S. Kumar, Contributions to Hardware and Software Reliability, World Scientific, Singapore, 1999.
K. Pillai and V.S.S. Nair, “A Model for Software Development effort and Cost Estimation”, IEEE Transactions on Software Engineering; vol. 23(8), pp. 485-497, 1997.
Kapur P. K, Singh V. B, Anand Sameer and Yadavalli V.S.S.,“Software Reliability Growth Model with Change-Point and Effort Control Using a Power Function of Testing Time” International Journal of Production Research, Vol. 46 Issue no. 3 pp.771-787,2008.
Kapur P. K, Singh V. B, Anand Sameer “Effect of Change-Point on Software Reliability Growth Models Using Stochastic Differential Equations” published in the proceedings of 3rd International Conference on Reliability and Safety Engineering, (Eds. R.B.Misra, V.N.A. Naikan, S.K.Chaturvedi, and N.K.Goyal), (INCRESE-2007), Udaipur, pp. 320-333, 2007.
Kapur P. K., Pham, H., Gupta, A. & Jha, P. C. Software Reliability Assessment with OR Applications, UK, Springer, 2011.
Kapur P.K. Kumar Archana ,Yadav Kalpana and Khatri Sunil “Software Reliability Growth Modelling for Errors of Different Severity using Change Point” International Journal of Quality ,Reliability and Safety Engineering ,Vol.14,No.4, pp 311-326,2007.
Kapur P.K., Garg R.B. A software reliability growth model for an error-removal phenomenon, Software Engineering Journal, July. Pp. 291-294, 1992.
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 57
Kapur P.K., Goswami D.N., Bardhan a.K., Singh Ompal, “Flexible Software reliability Growth Model with testing effort Dependent Learning Process” applied Mathematical Modelling, Vol. 32,No. 7 pp. 1298-1307, 2008.
Kapur P.K., Gupta A., Kumar A., Yamada S.; “Flexible Software Reliability Growth Models for Distributed Systems” OPSEARCH, 42 (4): 378-398, 2005.
Kapur P.K., Younes S., Agarwala S.; “Generalised Erlang Model with n types of faults”, ASOR Bulletin 14 pp. 5-11, 1995.
Kapur PK, Bardhan AK and Shatnawi O, “Why Software Reliability Growth Modelling Should Define Errors of Different Severity”, Journal of the Indian Statistical Association (Special Issue on Reliability Theory and Its Applications) 40(2), pp. 119-143, 2002.
Kapur PK, Bardhan AK, Kumar S. “On Categorization Of Errors In A Software” International Journal of Management and System 16(1): pp. 37-48, 2006.
Kapur PK, Bhusan S, Younes S. “An Exponential Software Reliability Growth Model with Upper Bound on Number of Failures” Microelectronics and Reliability, 33(9): pp. 1245-1249, 1993.
Kapur PK, Singh Ompal, Gupta Amit, “Some Modelling Peculiarities in Software Reliability” in Quality, Reliability and IT (Trends & future Directions), Eds.: P.K.Kapur and A.K Verma, Narosa Publications Pvt. Ltd., New Delhi 2005.
Kapur PK, Yadavalli VSS, and Gupta Anu “Software Reliability Growth Modeling Using Power Function Of Testing Time” International Journal of Operations and Quantitative Management, Vol.12, No.2, June 2006.
Kapur PK, Younes S, Sehgal VK and Yadav HS. “Discrete Imperfect Debugging Software Reliability Growth Model” Bulletin of Pure and Applied Sciences 15 E (2), pp 121-126, 1996.
Kapur, P.K. and Younes, S. “Software reliability growth model with error dependency”, Microelectronics and Reliability, Vol. 35 No. 2, pp. 273-278, 1995.
Kapur, P.K., Gupta Amit, Gupta Anu, Kumar A., Discrete Software Reliability Growth Modelling, Published in “Quality, Reliability and IT (Trends & Future Directions)”, Eds.: P.K. Kapur and A. K. Verma, Narora Publications Pvt.Ltd., New Delhi, 2005.
Kapur, P.K., Xie, M., Garg, R.B. and Jha, A.K. A discrete software reliability growth model with testing effort, Published in the Proceedings of 1st International Conference on Software Testing, Reliability and Quality Assurance (STRQA), held during Dec 21-22 at Hyatt Regency, New Delhi, 16-20, 1994.
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 58
Khoshgoftaar TM “Non-Homogenous Poisson Process for Software Reliability Growth”, COMPSTAT’88, Copenhagen: Denmark, pp. 13-14, 1988.
Kimura M, Yamada S and Osaki S, “Software Reliability Assessment for an Exponential S-shaped Reliability Growth Phenomenon”, Computers and Mathematics with Applications 24, pp. 71-78, 1992.
Kumar Archana, Ph.D. Thesis “A Study in Software Reliability Growth Modelling Under Distributed Development Environment”, Department of Operational Research, University of Delhi, Delhi, 2007.
Littlewood B. “Stochastic Reliability Growth: A Model for Fault Removal in Computer Program and Hardware Design”, IEEE Transactions on Reliability; R-30, No. 4, pp.313-320, 1981.
Littlewood, B. “Modeling Growth in Software Reliability. In Software Reliability Handbook” Ed Paul Rook. Elsevier Applied Science, pp. 111-136, 1991.
Musa JD, Software Reliability Engineering, McGraw-Hill, 1999.
Musa J.D. “Validity of execution time theory of software Reliability” IEEE Transactions on Reliability, R-28: pp 181-191, 1979.
Musa J.D., A. Iannino and K. Okumoto, Software Reliability: Measurement, Prediction, Applications, McGraw Hill, 1987.
Musa JD “A Theory of Software Reliability and Its Applications”, IEEE Transactions on Software Engineering; SE-1, pp. 312-327, 1975.
Musa JD and Okumoto “A Logarithmic Poisson Execution Time Model For Software Reliability Measurements”, Proceedings of the 7th International Conference on Software Engineering, Orlando, F1, pp. 230-237, 1984.
N. Langberg and N.D. Singpurwalla, “A Unification of Some Software Reliability Models,” SIAM J. Scientific and Statistical Computing, vol. 6, no. 3, pp. 781-790, Mar. 1985.
Ohba M and Yamada S “S-shaped Software Reliability Growth Model”, Proceedings of the 4th International Conference on Reliability and Maintainability, pp.430-436, 1984.
Ohba M., “ Software Reliability Analysis Models”, IBM Journal of Research and Development, Vol. 28, pp. 428-443, 1984.
Ohba M., “Inflection S-Shaped Software Reliability Growth Models. Osaki, S. and Hatoyama, Y. (Eds.), Stochastic Models in Reliability Theory, Berlin, Germany: Springer-Varlag, pp. 44-162, 1984.
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 59
Pham H. “Software Reliability” Springer-Verlag Singapore, 2000.
Pham H. and X. Zhang, “An NHPP Software Reliability Models and its Comparison “International Journal of Reliability, Quality and Safety Engineering, 4, pp. 269-282, 1997.
Pham H., System Software Reliability, Reliability Engineering Series Springer, 2006.
Pham, “Hanbok of Engineering statistics” Springer, 2006(a).
Popstajanova, K. and K. Trivedi: “Architecture Based approach to Reliability Assessment of Software Systems”, Performance Evaluation, Vol. 45, No.2, pp.179-204, 2001.
Pressman RS, Software Engineering: A Practitioner’s Approach, McGraw-Hill: Fifth Edition, 2001.
Schneidewind N.F., Modelling the fault correction process, Proceedings of the 12th International Symposium on Software Reliability Engineering, IEEE Computer Society Press: Los Alamitos, CA, pp. 185–190, 2001.
Shanthikumar, J. G. “A general software reliability model for performance prediction”, Microelectronics Reliability, Vol. 21, pp. 671–682, 1981.
Shanthikumar, J. G. “Software reliability models: a review”, Microelectronics Reliability, Vol. 23, pp. 903–943, 1983.
Singh V.B. “A Study on Software Reliability Growth Model using Change-Point Fault Dependency” Ph.D. Thesis, University of Delhi, Delhi, 2008.
Singh, V.B., Yadav, K., Kapur, R. and Yadavalli, V.S.S. “Considering fault dependency concept with debugging time lag in software reliability growth modeling using a power function of testing time”, International Journal of Automation and Computing, Vol. 04 No. 4, pp. 359-368, 2007.Singpurwalla ND, Wilson SP. “Statistical Methods in Software Engineering – Reliability and Risk” New York: Springer-Verlag, 1999.
Singpurwalla ND. “The Failure Rate Of Software: Does It Exist?” IEEE Transactions on Reliability 44(3): pp.463-469, 1995.
Wood A. “Predicting Software Reliability” IEEE Computers, 11: pp. 69-77, 1996.
Xie M. “Software Reliability Modelling” World Scientific, 1991.
Xie M., Zhao M., The Schneidewind software reliability model revisited, In Proceedings of the 3rd International Symposium on Software Reliability Engineering, pp. 184–192, 1992.
A Study of Discrete and Continuous Software Reliability Growth Models
Department of Mathematics, DNPG College , Gulaothi, Bulandshahar C.C.S University Meerut 60
Yamada S, Ohtera H, and Narithisa H. “Software Reliability Growth Models With Testing Effort” IEEE Trans. On Reliability, R-35 (1): pp. 19-23, 1986.
Yamada S., Kimura M., Tanaka H., Osaki S., Software reliability measurement and assessment with stochastic differential equations, IEICE Transactions on Fundamentals of Electronics and Computer Sciences, Vol. E77-A, No. 1, pp. 109-116, 1994.
Yamada S., Nishigaki A., Kimura M., A Stochastic Differential Equation Model for Software Reliability Assessment and its Goodness of Fit, International Journal of Reliability and Applications, Vol. 4, No. 1, pp. 1-11, 2003.
Yamada S., Obha M. and Osaki S., “S-shaped software reliability growth modelling for software error detection”, IEEE Trans. On Reliability, Vol. R-32 No. 5, pp. 475-484, 1983.
Yamada S., Ohba M., Osaki S.; “S-Shaped Software Reliability Growth Models and their Applications”, IEEE Transactions on Reliability; R-33: 169-175, 1984.
Yamada S., Osaki S. Software reliability growth models”, Applied- Stochastic Models and Data analysis,1,65-77, 1985.
Yamada S., Tamura Y and Kimura M.; A Software Reliability Growth Model For a Distributed Development Environment-Electronics & Communication in Japan Part 3, Vol. 83, pp 46-53, 2000.
Yamada S., Tamura Y., A Flexible Stochastic Differential Equation Model in Distributed Development Environment , European Journal of Operational Research, Vol. 168, pp. 143-152,2006.
Yamada, S., M. Obha and S. Osaki, S-Shaped Software Reliability Growth Modeling For Software Error Detection, IEEE Trans. On Reliability, Vol. R-32, No. 5, pp. 475-484, 1983.
Younes S. “Studies in Software Reliability Modelling–Ph.D. Thesis” University of Delhi, Delhi, 1990.
Zhang X and Pham H.” An Analysis of Factors Affecting Software Reliability “ Journal of Systems and Software, Vol. 50 No.1, pp. 43-56(14), 2000.
Zhao, M “Change-Point Problems In Software And Hardware Reliability”, Communications in Statistics-Theory and Methods, 22(3), pp. 757-768, 1993.