1

Software Testing, 2nd

edition
NARESH CHAUHAN

© Oxford University Press 2017. All rights reserved.

 2

Chapter 2
Software Testing Terminology
and Methodology

© Oxford University Press 2017. All rights reserved.

 Objectives
Difference between error, fault and failure 1

Life cycle of a bug

How bug affects economics of software testing

How bugs are classified

Testing principles

Software testing life cycle (STLC) and its models

Difference between verification and validation

Development of software testing methodology

© Oxford University Press 2017. All rights reserved.
 Software Testing Terminology 2

Failure
The inability of a system or a component to perform a required function
according to its specification.

Fault / Defect / Bug

Fault is a condition that causes a system to produce failure. It can be said that
failures are manifestation of bugs.

© Oxford University Press 2017. All rights reserved.

 Software Testing Terminology
3
Error
Whenever a member of development team makes any mistake in any phase of
SDLC, errors are produced. It might be a typographical error, a misleading
specification, a misunderstanding of what a subroutine does, and so on. Thus,
error is a very general term used for human mistakes.

© Oxford University Press 2017. All rights reserved.

 Software Testing Terminology 4
Test Case
It is a well-documented procedure designed to test the functionality of a feature in
the system. A test case has an identity and is associated with a program
behaviour. The primary purpose of designing a test case is to find errors in the
system. A test case needs to specify a set of inputs and the corresponding
expected outputs. The sample for a test case is shown in the figure.

© Oxford University Press 2017. All rights reserved.

 Software Testing Terminology
5

Testware
The documents created during the testing activities are known as
Testware.

Incident
The symptom(s) associated with a failure that alerts the user to the
occurrence of a failure.

Test Oracle
It is the means to judge the success or failure of a test.

© Oxford University Press 2017. All rights reserved.

 Life Cycle of a Bug
6

© Oxford University Press 2017. All rights reserved.

 States of a Bug
7

© Oxford University Press 2017. All rights reserved.

Bugs Affect Economics of Software
8
Testing

© Oxford University Press 2017. All rights reserved.

Bug Classification based on Criticality 9
Critical Bugs
This type of bug has the worst effect as it stops or hangs the normal
functioning of the software.

Major Bug
This type of bug does not stop the functioning of the software but it causes a
functionality to fail to meet its requirements as expected.

Medium Bugs
These are less critical in nature as compared to critical and major bugs.

Minor Bugs
These are just mild bugs, which occur without any effect on the expected
behavior or continuity of the software.

© Oxford University Press 2017. All rights reserved.

Bug Classification based on SDLC 10
Requirements and Specifications Bugs
Design Bugs
❑ Control Flow Bugs
❑ Logic Bugs
❑ Processing Bugs
❑ Data Flow Bugs
❑ Error Handling Bugs
❑ Race Condition Bugs
❑ Boundary Related Bugs
❑ User Interface Bugs
Coding Bugs
Interface and Integration Bugs
System Bugs
Testing Bugs
© Oxford University Press 2017. All rights reserved.
 Testing Principles 11
Effective testing, not exhaustive testing.

Testing is not a single phase performed in SDLC.

Destructive approach for constructive testing.

Early testing is the best policy.

Probability of existence of an error in a section of a program is

proportional to the number of errors already found in that section.

Testing strategy should start at the smallest module level and

expand towards the whole program.

© Oxford University Press 2017. All rights reserved.

 Testing Principles
12

Testing should also be performed by an independent team. Everything must

be recorded in software testing.

Invalid inputs and unexpected behavior have a high probability of finding an

error.

Testers must participate in specification and design reviews.

© Oxford University Press 2017. All rights reserved.

Software Testing Life Cycle (STLC) 13

© Oxford University Press 2017. All rights reserved.

 Test Planning
Define the test strategy. 14

Estimate the number of test cases, their duration, and cost.

Plan the resources such as the manpower, tools required, documents

required.

Identify areas of risks.

Define the test completion criteria.

Identify methodologies, techniques and tools for various test cases.

Identify reporting procedures, bug classification, databases for testing,

bug severity levels, project metrics.
© Oxford University Press 2017. All rights reserved.
 Test Design
15
Determine the test objectives and their prioritization.
Prepare the list of items to be tested.
Map items to test cases.
Select test case design techniques.
Create test cases and test data.
Set up the test environment and supporting tools.
Create test procedure specification.

© Oxford University Press 2017. All rights reserved.

 Test Execution
16

© Oxford University Press 2017. All rights reserved.

 Post-Execution / Test Review
17

Understanding the bug

Reproducing the bug

Analysing the nature and cause of the bug

Reliability analysis

Coverage analysis

Overall defect analysis

© Oxford University Press 2017. All rights reserved.

Software Testing Methodology
18

© Oxford University Press 2017. All rights reserved.

 Test Strategy Matrix
Select and rank test factors 19

Identify the system development phases

Identify the risks associated with system under development

© Oxford University Press 2017. All rights reserved.

 Development of Test Strategy 20

Verification: “Are we building the product right?”

Validation: “Are we building the right product?”

© Oxford University Press 2017. All rights reserved.

V Testing Life Cycle Model 21

© Oxford University Press 2017. All rights reserved.

 Validation Activities 22

Unit testing

Integration testing

Function testing

System testing

Acceptance testing

© Oxford University Press 2017. All rights reserved.

 Testing Tactics 23
Testing tactics are the ways to perform various types of testing. This can be done in
two ways: manual testing and automated testing.

Manual testing is the type of testing performed when the technique is applied
manually, that is, no automation tool is used.

On the contrary, automated testing is performed with the help of testing to save
time and effort. To automate the test cases, sometimes, the tester may need to
write a set of routines.

© Oxford University Press 2017. All rights reserved.

Testing Tactics 24

© Oxford University Press 2017. All rights reserved.

 Testing Tactics 25
Actual methods for designing test cases, that is, software testing techniques,
implement the test cases on the software. These techniques can be categorized into:

Static testing: It is a technique for assessing the structural characteristics of source

code, design specifications or any notational representation that conforms to
well-defined syntactic rules. It is called as static because we never execute the code
in this technique.

Dynamic testing: In this technique, the code is run on a number of inputs provided
by the user and the corresponding results are checked. This type of testing is further
divided into two parts: (a) black-box testing and (b) white-box testing.

© Oxford University Press 2017. All rights reserved.

Testing Tactics 26

© Oxford University Press 2017. All rights reserved.

Testing Tactics 27

© Oxford University Press 2017. All rights reserved.