-Introduction to Software Reliability-

By Priya Singh (Assistant Professor, Dept of SE, DTU)

1. Definition
The probability of failure-free operation of a software component or
system in a specific environment for a specified period of time.

It is possible to estimate reliability for each severity of failures (critical,

major, minor, say) separately.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

2. Hardware Reliability vs Software
Reliability
• Hardware devices can become inoperative due to three types of
causes- design errors, manufacturing errors, and failure due to wear
and tear.
• Software unreliability is entirely attributable to design errors.
• The hardware failure follows the well-known bathtub curve. The
failure rates are high during the infant mortality period and wear out
period in the bathtub curve. However, the software does not wear out.
• The failures in software systems occur not due to fatigue, but due to
exercising that portion of the code which container fault.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

The main differences between hardware reliability and software
reliability are given below-

By Priya Singh (Assistant Professor, Dept of SE, DTU)

3. Need for Software Reliability
• The reliability of many modern • Failure of consumer durables or PC
applications depends on the reliability systems can be annoying as well as
of the software systems in them. affect productivity.
• Consumer durables, such as toys • The reliability of the software system
microwave once, mobile phones, should therefore be adequate and
television receivers, etc. contain commensurate with the needs of the
software and therefore correct user and hence it is important to
performance of both the hardware and measure and quantify it.
software is essential to use them.
• The reliability of software systems in
safety-critical applications, such as
nuclear power plants, space shuttles, etc.
is quite critical since the loss of mission
can lead to catastrophic results.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• The reliability of hardware is • Software reliability engineering is a major
improving due to maturity in design area of concern and focus in the 21
and manufacturing processes. century.
• Reliability of software is a major • According to Musa, “Software reliability
area of concern since our discipline is the most important in the most
is still nascent. measurable aspect of software quality, and
• Software-driven outages exceed it is very much customer-oriented.
hardware outages by a factor of 10. • The increasing global competition and
• The ability to deliver reliable high development costs have necessitated
computer hardware can be quantifying software quality in terms of
considered to be given. reliability achieved and to measure and
control the level of quality delivered.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

4. Categories of Software Reliability Model
Software reliability models can be classified as given below, depending
on the type of data used to estimate reliability,
#1 Predictive Models
#2 Estimator Models

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• The predictive models arrive at reliability
estimates of the software early in the SDLC
even before coding begins. • The estimator models use data from
• Such models predict future reliability based on the current project, and reliability
historical data collected in the organization from estimates are carried out with
previous projects, expected properties of failure data collected during testing
software products- LOC, function points,
complexity measures, etc., and hence the and or operation.
reliability can be modeled prior to the • Estimator models estimate future
commencement of development of software
systems. reliability based on past failure
• Predictive models may predict future reliability data obtained from the current
based on experience from the past projects project. Hence, the estimates tend
executed since the development. The to be more accurate.
environment in the current project in terms of
requirements, technology, resources etc may be
different. Outcome of the current project based
on past project may tend to be inaccurate.
By Priya Singh (Assistant Professor, Dept of SE, DTU)
4.1 Predictive Model
• Predictive models estimate the number of faults that will be present in a software system,
failure rate and consequently the reliability of the system starts before development.
• In fact, Failure Intensity is another measure of reliability, and for some software systems,
it is convenient to express reliability in terms of Failure Intensity at a given point of time.
It is also known as Rate of Occurrence Of Failure, ROCOF or failure rate- the average
number of failures per unit time at a given time/duration.
• It will keep on varying with time like reliability. If failure intensity decreases, reliability
will increase and vice versa.
• Failure intensity is not probabilistic.
• We can arrive at a point estimate for failure intensity. If required, the point estimate can
be converted into a range estimate. For instance, in the point estimate, we may say that
the failure intensity after 100 hours of testing will be 0.002 failures/hour. In range
estimate, the same with a certain amount of confidence level is reported. For instance, we
may say that failure intensity after 100 hours of testing will lie between 0.001-0.004
failures/hour at a confidence level of 90%. At various levels of confidence 85%, 95% etc.,
the range of failure intensity for a specified point estimate will vary.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

E.g. MUSA Model

By Priya Singh (Assistant Professor, Dept of SE, DTU)

4.2 Estimator Model
• The estimator models arrive at future reliability based on data on software failures
observed in the early part of the current project itself.
• Since the estimate is based on actual failures of the same product in the early
stages, the estimate is likely to be more accurate.
• Software reliability can be estimated independently in different phases of SDLC.
• For instance, reliability estimated during the testing phase is known as testing
reliability and the reliability estimated during the operation and maintenance phase
of SDLC is called operational reliability.
• The software reliability has to be estimated in the testing phase and operational
phase of SDLC independently since the environment during these phases is
usually different. For instance, white faults causing failure may be corrected
during the testing phase. It may not be usually so during the operational phase.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

 4.2.1 Operational reliability
• Operational phase begins after the termination of the testing phase in the SDLC
and the development agency releases the product for actual use of the customer. It
commences after the completion of acceptance test.
• Definition- The probability of no failure occurred during a specified time interval
in the operational phase.
• During the operational phase of software systems, usually the failures that are
noticed are reported, but the faults causing those failures are however not
corrected.
• The fault fixing schedule generally is postponed to the next version of the
software.
• It is suggested by Musa not to count the repeated occurrences of the same failure.
• It is also known as Certification Model for Reliability as it determines reliability
achievable during actual use of this software systems.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

4.2.2 Testing reliability
• This type of model is applied during the testing phase when failures are observed and the
faults causing failures are fixed.
• Testing reliability is defined as the probability of no failure occurring during a specified
time interval in the testing phase.
• During the system testing phase, the final testing activity after integration of the software
subsystems, the faults causing failures are corrected.
• The Test Analyze and Fix (TAF) procedure is continuously followed during this phase.
• If it is assumed that correction of faults does not introduce any new fault, the probability
that no failure occurs during a fixed time interval, which is the same as reliability,
increases with the progress of software testing.
• The failure intensity ultimately reduces and reliability improves as faults are detected and
corrected.
• While during the operational phase, the user will experience a constant failure rate since
there is no fault removal, the failure rate will vary with testing time in the testing phase
where identified faults are removed and similar failures will not occur again.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

5. Black box and white box models
• Software testing can be classified as white-box testing and black-box testing depending upon the
methodology used for test case design.
• In black-box testing, test cases are derived by knowing the specified functions a product has been designed to
perform.
• On the contrary, in white-box testing, the test case is derived by knowing the internal structure of the
workings of the product.
• Black-box software reliability models treat software as a black-box. Such a model does not require any
knowledge of the internal structure of the software.
• Opposite to it, the white-box models are obtained with full knowledge of the internal structure of the software,
and by studying the coverage of the software by test data. The various coverage criteria for white-box testing
includes-
#1 Statement coverage
#2 Decision coverage
#3 Condition coverage
#4 Condition Decision coverage
#5 Multiple Condition coverage
An estimate of reliability will be made in the white-box software reliability model by running test cases,
observing coverage of the program blocks as above, and the percentage of test cases passing. Traditionally in
quality and reliability analysis, item under examination is considered to be a black box and black box models
will interest the user more.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

6. System testing
• System testing carried out towards the end of SDLC when the complete
program has been integrated, is aimed at validating the product.
• According to Musa and Ackerman, for any application, the number of
distinct input combinations you need to verify is enormous. Testing is
therefore impractical and that is the reason no software system of any
realistic size is ever completely debugged, i.e. error-free.
• Software Reliability Engineering provides a methodology based on
statistics to quantify with a given confidence level the probability of a
number of hours/day for the failure-free operation.
• According to Musa and Ackerman, applying reliability measurement during
system testing in SDLC provides quantitative information on the validation
process. System test is thus for twin purposes-
#1 To validate the completed software and
#2 To improve reliability of software

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• According to Myers, testing is a process of executing a program with
the goal of finding errors.
• The purpose of software system testing prior to release of the product
is also to execute the software system and look for failures.
• Whenever a failure occurs and effort is made to identify the fault in the
system under test (SUT) and correct it.
• During system testing, selecting test cases according to the operational
profiles is more efficient with regard to reduction of failure intensity
since it reveals those failures that the system user is most likely to
encounter in actual use.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

7. Software Reliability Growth Models
(SRGM)
• One of the objectives of system testing is to collect as many faults as possible
without introducing any additional faults in the process.
• The fault correction process should reduce the failure rate and thereby improve the
reliability of SUT. Hence, this process is known as Software Reliability Growth
Testing (SRGT).
• During SRGT, the failure rate, also known as ROCOF as well as failure intensity
function changes with time.
• The total rate of failure occurrence that is a failure intensity of the software system
is more important to measure in the context of software reliability and analysis.
The failure intensity function of SUT is influenced by the following-
1. Complexity of SUT
2. Number of testers deployed
3. Skill levels of testers
4. Type of testing performed (e.g., black box testing, white box testing)
By Priya Singh (Assistant Professor, Dept of SE, DTU)
• The various reliability models address through different approaches,
the variation of failure intensity function during SRGT on account of
observation of failures during execution of the software and correction
of faults causing the failure.
• The category of Software Reliability Models used for modeling
reliability growth during SRGT are popularly known as Software
Reliability Growth Models (SRGM) as they help in estimation of the
growth of reliability with the testing due to removal of faults in the
SUT.
• The SRGM facilitates engineers and managers in tracking and
measuring the growth of reliability as software is being improved.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Software reliability figures are used to evaluate alternative or
competing software technologies.
• For instance, to evaluate the gains of software tool-based testing, the
quantum of reduction of failure intensity with the same testing effort
or specified hours of testing may be compared.
• Between manual testing and software tool-based testing, the following
are some common measures used to express software reliability.
1. ROCOF
2. Mean Time To Failure
3. Probability of Failure-Free Operation of the Software System
for specified time.
4. Residual Fault Density.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

8. Musa's prediction model for failure
intensity
• John D. Musa has contributed immensely to the development of software reliability engineering as a unique
and useful discipline internationally.
• He has outlined a predictive model for predicting the initial failure intensity before the start of the system
tests.
• The failure intensity before start of the system testing indicates the quality of development.
• It indicates the quality and reliability of the system under test.
• It is useful in estimating the quantum of test effort needed.

This predictive model is described as-

By Priya Singh (Assistant Professor, Dept of SE, DTU)

• Musa is given a methodology to derive each of the terms in equation 11, which are described
below-

By Priya Singh (Assistant Professor, Dept of SE, DTU)

The above equation can be used to predict failure intensity before start
of system testing.
The accuracy of the prediction depends upon the accuracy of five
factors in equation 1.5.
It gives some idea about the software development project in advance.

By Priya Singh (Assistant Professor, Dept of SE, DTU)

9. Operational reliability model
• During the operational phase, the reliability of this software at any
time will follow exponential distribution as given below-

• where λ is the failure rate, and

t is the operational time computed from the release of the software for
operations. It is constant and is the reciprocal of mean time between failures
(MTBF).

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Example 1

Tip:
MTBF= Average of TBF as given in table
Failure rate = 1/MTBF

By Priya Singh (Assistant Professor, Dept of SE, DTU)

Example 2

By Priya Singh (Assistant Professor, Dept of SE, DTU)