aaron hoff. overview compare and hardware and software reliability discuss why software should be...
Post on 24-Dec-2015
218 Views
Preview:
TRANSCRIPT
Software Reliability
Aaron Hoff
OverviewCompare and hardware and software
reliabilityDiscuss why software should be reliable?Describe MLE (Maximum Likelihood
Estimation)Show two specific reliability models
Mill’s Error Seeding ModelJelinski-Moranda Model
Software Reliability ToolsTrainingConclusion
ReliabilityWebster’s Dictionary defines reliability as:
1. The quality or state of being suitable or fit to be relied on, dependable2. The extent to which an experiment, test, or measuring procedure yields the same result on repeated trials.
Hardware Reliability
Hardware Reliability(cont.)Failure rate is very high during the burn in
period.Many faults are found on all components.Thorough testing of all components cuts
down on the number of faults.Enters the useful life with small amount of
faults.After time, wears down and quickly increases
in failure rate.
Software Reliability
Software Reliability(cont.)Starts with many faults in system when first
createdAfter much testing/debugging enters useful
life cycleUseful life includes upgrades made to system
which bring about new faults.System needs to then be tested to reduce
faults.Eventually evens out into the Obsolescence
cycle where software is usually quite reliable.
Compare/Contrast HW & SWBoth start out with large number of faultsBoth need to be tested completely
Hardware faults are physical, software faults are not
Hardware stays at steady reliability level in useful life, software needs constant testing after upgrades.
Hardware wears out over time, software does not.
Hardware failure is random, software failure is systematic.
Why should software be reliable?Examples of software failure
Therac 25 (1985 – 1986)Ariane 5 (1996)Mars Lander (1999)
Lives can be put in dangerMoney and time can be lostLost trust of customers.
Software Reliability ModelsWhy do we need them?
Predict probability of failure of a component or system
Estimate the mean time to the next failure Predict number of (remaining) failures
Software Reliability ModelsMany models have come out to help increase
reliabilityAll models can be grouped into these
categories: Error seeding, Failure rate, Curve fitting, Reliability growth, Program structure, Input domain, Execution path, Nonhomogenous Poisson process, Bayesian and unified, and Markov.
MLE(Maximum likelihood estimation) is method used for fitting statistical model to data
MLEPrinciple is to estimate parameter values of
models which make observed data most likely to occur
Probability itself cannot be used for estimation of parameters
MLE uses likelihood instead to estimate parameter values
Uses sample data set to estimate different parameter values.
Coin Flip ExampleProbability of flipping heads over 100 flips is
0.5Say sample set is: Heads was flipped 56 times
Plug variety of probability values into this model to obtain data that can be graphed.
Graph for MLE of Coin Flip example
Error SeedingEstimates total number of errors by
introducing known errors into software.Terminology
Inherent error – one found in software that causes failure regardless of what user does.
Induced(seeded) error – one intentionally inserted into piece of software to estimate total number of errors
Mill’s error seeding modelProposed in 1970 by MillUsed during testing phaseDevelopers/Testers insert errors in places
they think errors would occur (error prone locations)
Test whole systemGather data on all errors found during testing
processTotal number of errors can then be estimated
Mill’s Error Seeding Model(cont.)Hypergeometric distribution to find probability of k induced
errors
where N = total number of inherent errorsn1 = total number of induced errorsr = total number of errors removed during debuggingk = total number of induced errors in r removed errorsr – k = total number of inherent errors in r removed errors
Mill’s Error Seeding Model(cont.)The hypergeometric distribution can be
simplified greatly to achieve an equation for total # of inherent errors
The lower the total number of errors estimated, the higher the reliability.
Advantages/DisadvantagesA - The fault representation is easy to apply
with fault-seeding tool.A - Can be used to predict the fault
distribution of particular software.
D - This model is very time consumingD - Can not be applicable in large programsD - Always chance of human error when
deciding where to put all induced error.
Failure Rate(Terminology)Failure Rate – the frequency with which an
engineered system or component fails.Failure – Occurs when the user perceives that
the program ceases to deliver the expected service
Fault – The cause of the failure or internal error of software
Basic premise of failure rate is that successive failure rates will get longer as faults are removed from the software system.
Jelinski-Moranda ModelOne of the earliest models(1972) proposed when
looking into software reliability.Six assumptions:
Program contains N initial faults which is unknown but a fixed constant
Each fault is independent and equally likely to cause failure
Time intervals between failures are independentWhen failure occurs, corresponding fault is removedFault is assumed to be instantaneously removed, no new
faults are inserted during removalSoftware failure rate is constant and proportional to
number of faults remaining in software
Jelinski-Moranda ModelThe six assumptions set ground rules for
model.Intensity fail rate function:
where φ = a proportional constant, the contribution
any one fault makes to the overall program;N = the number of initial faults in the
program;ti = the time between the (i-1)th and the ith
failures.
Jelinski-Moranda ModelThe intensity function is used to obtain magnitude
of the failure rate at a certain failure interval.Infer: After first failure rate where fault is removed
with certainty, intensity shall be lowered in proportion to number of faults remaining.
Jelinski-Moranda used this information to obtain a reliability function.
MLE is used to estimate values such as the number of initial faults(N) or proportional constant(φ)
Software Reliability ToolsSMERFS(Statistical Modeling and Estimation
of Reliability Functions for Software) - allows user to perform complete software reliability analysis
SARA(Software Assurance Reliability Automation) - incorporates both reliability growth modeling and design code metrics for analyzing software time between failure data
TrainingTraining organizations
RACReliaSoftSoftRel - www.softrel.com/prod03.htmSoHaR
ConclusionSW reliability is similar to HW reliability but
must be treated differentlyReliability of software is something to strive
forCan prevent major faults that have possibility
of taking human life, money, time, and customers
Useful to have model or something to measure
Many models have been proposedTraining is available to those who want to
learn more about reliability engineering
References[1] Ensuring Software Reliability. Neufelder, Ann Marie. New York: Marcel
Dekker, Inc., 1992. 1-242.
[2] Goddard Space Flight Center. Overview of Software Reliability. February 16,
2005. <http://sw-assurance.gsfc.nasa.gov/disciplines/reliability/index.php> [3] Handbook of Software Reliability Engineering. Ed. Michael R. Lyu.
New York: McGraw-Hill Companies, Inc., 1996. 3-779. [4] Lloyd, Robin. Metric mishap caused loss of NASA orbiter. September
30,1999. <http://cnn.com/TECH/space/9909/30/mars.metric.02/>
[5] Pan, Jiantao. Software Reliability. Spring 1999.
<http://www.ece.cmu.edu/~koopman/des_s99/sw_reliability/#metrics>
References(cont.)[6] Purcell, S. Maximum Likelihood Estimation. May 20, 2007.
<http://statgen.iop.kcl.ac.uk/bgim/mle/sslike_1.html> [7] Software Reliability. Pham, Hoang. Singapore: Springer-Verlag Singapore
Pte. Ltd., 2000. 1-219. [8] Software Reliability and Testing. Pham, Hoang. Piscataway: The Institute of
Electrical and Electronics engineers, Inc., 1995. 1-133. [9] Software Reliability Models. Malaiya, Yashwant K., and Pradip K. Srimani.
New York: Institute of Electrical and Electronics Engineers, Inc., 1990. 1-121.
[10] Software Safety and Reliability. Herrmann, Debra S. Piscataway: The
Institute of Electrical and Electronics engineers, Inc., 1999. 5-466.
top related