Introduction to Software Testing

Topic: Introduction to Software Testing

Presentation Outline

Evolution of Software Testing

- What is Software Testing?

- Definitions
- Testing Objectives
- Who Does Software Testing?

- Software Testing Activities

- Software Testing Scope

- Software Testing Principles

- Software Testing Process

- Software Testing Myths

- Software Testing Limits

- Different Types of Software Testing

Evolution of Software Testing
 Topic: Introduction to Software Testing

What is Software Testing

Several definitions:

“Testing is the process of establishing confidence that a program or system

does what it is supposed to.” by Hetzel 1973

“Testing is the process of executing a program or system with the intent of

finding errors.” by Myers 1979

“Testing is any activity aimed at evaluating an attribute or capability of a

program or system and determining that it meets its required results.”
by Hetzel 1983

What is IEEE’s definition?

Software testing is the process of analyzing a software item to detect the

differences between existing and required conditions (that is, bugs) and
to evaluate the features of the software item
 Topic: Introduction to Software Testing

What is Software Testing

- One of very important software development phases

- A software process based on well-defined software

quality control and testing standards, testing methods,
strategy, test criteria, and tools.

- Engineers perform all types of software testing

activities to perform a software
test process.

- The last quality checking point for software on its

production line
 Topic: Introduction to Software Testing

Who does Software Testing

- Test manager
- manage and control a software test project
- supervise test engineers
- define and specify a test plan

- Software Test Engineers and Testers

- define test cases, write test specifications, run tests

- Independent Test Group

- Development Engineers
- Only perform unit tests and integration tests

- Quality Assurance Group and Engineers

- Perform system testing
- Define software testing standards and quality
control process
 Topic: Introduction to Software Testing

Software Testing Scope

Software Testing Software Testing Software Testing

Management Methods Strategies

Configuration Software Testing Software Test

Management Process Models

Software Problem Software Testing Software Test

Management Tools Criteria
 Topic: Introduction to Software Testing

Software Testing Activities

- Test Planing
Define a software test plan by specifying:
- a test schedule for a test process and its activities, as
well as assignments
- test requirements and items
- test strategy and supporting tools

- Test Design and Specification

- Conduct software design based well-defined test
generation methods.
- Specify test cases to achieve a targeted test coverage.

- Test Set up:

- Testing Tools and Environment Set-up
- Test Suite Set-up

- Test Operation and Execution

- Run test cases manually or automatically
 Topic: Introduction to Software Testing

Software Testing Activities

- Test Result Analysis and Reporting

Report software testing results and conduct test result analysis

- Problem Reporting
Report program errors using a systematic solution.

- Test Management and Measurement

Manage software testing activities, control testing schedule, measure
testing complexity and cost

- Test Automation
- Define and develop software test tools
- Adopt and use software test tools
- Write software test scripts and facility

- Test Configuration Management

- Manage and maintain different versions of software test suites, test
environment and tools, and documents for various product versions.
 GOALS OF SOFTWARE TESTING

1. Short-term or immediate goals of software testing: - These goals are the immediate results after performing testing.
These goals even may be set in the individual phases of SDLC. Some of them are completely discussed below:

a) Bug discovery: The immediate goal about software testing is to find errors at any stage of software development. More
the bugs discovered at early stage, better will be the success rate about software testing.

b) Bug prevention: It is the consequent action of bug discovery. From the behavior and analysis of bugs discovered,
everyone in the software development team gets to learn how to code so that bugs discovered should not be repeated in later
stages or future projects.

2. Long-term goals of software testing: - These goals affect the product quality in the deep run, when one cycle of the
SDLC is over. Some of them are completely discussed below:

a) Quality: Since software is also a product, so its quality is primary from the user’s point of view. Thorough testing
ensures superior quality.

Quality depends on various factors, such as correctness, integrity, efficiency, and reliability. So to achieve quality you have
to achieve all the above mentioned factors of Quality.

b) Customer satisfaction: From the user’s perspective, the prime goal of software testing is customer satisfaction only. If
we want the client and customer to be satisfied with the software product, then testing should be complete and thorough.
Post-implementation goals of software testing: - These goals are become essential after the product is released. Some of
them are completely discussed below:

a) Reduced maintenance cost: The maintenance cost about any software product is not its physical cost, as effective
software does not wear out. The only maintenance cost in a software product is its failure due to errors.

Post- release errors are always costlier to fix, as they are difficult to detect. Thus, iftesting has been done rigorously and
effectively, then the chances about failure are minimized and as a result of this maintenance cost is reduced.

b) Improved software testing process: A testing process for one project may not be blooming successful and there may
be area for improvement. Therefore, the bug history and post-implementation results can be analyzed to find out snags in
the present testing process, which can be determine in future projects.
 Goals Summary
Though errors always cannot be prevented to zero, they can be minimized. In this sense
prevention of a bug is a superior goal of testing.

A complete testing process achieves reliability, reliability enhances the quality, and quality
in turn, increases the customer satisfaction.

The long-term post-implementation goal is to improve the testing process for future
projects.
 Topic: Introduction to Software Testing

Software Testing Principles

•Principle #1: Complete testing is impossible.

•Principle #2: Software testing is not simple.

•Reasons:
•Quality testing requires testers to understand a system/product completely
•Quality testing needs adequate test set, and efficient testing methods
•A very tight schedule and lack of test tools.

•Principle #3: Testing is risk-based.

•Principle #4: Testing must be planned.

•Principle #5: Testing requires independence.

•Principle #6: Quality software testing depends on:

•Good understanding of software products and related domain application
•Cost-effective testing methodology, coverage, test methods, and tools.
•Good engineers with creativity, and solid software testing experience
 Psychology of Testing
• A program is its programmer’s baby!
– Trying to find errors in one’s own program is like
trying to find defects in one’s own baby.
– It is best to have someone other than the
programmer doing the testing.
• Tester must be highly skilled, experienced
professional.
• It helps if he or she possesses a diabolical mind.
 Psychology of Testing
• Testing achievements depend a lot on what are
the goals.
• Myers says (79):
– If your goal is to show absence of errors, you will
not discover many.
– If you are trying to show the program correct, your
subconscious will manufacture safe test cases.
– If your goal is to show presence of errors, you will
discover large percentage of them.

Testing is the process of executing a program with the

 In Psychology of testing Tester and others can
improve communication by
Starting collaboration instead of battles.
Everyone should have common goal of the quality
software.
Improve mutual understanding while working.
Confirm that the other person has understood what
you have said.
 Topic: Introduction to Software Testing

Software Testing Myths

- We can test a program completely. In other words, we test a program exhaustively.

- We can find all program errors as long as test engineers do a good job.

- We can test a program by trying all possible inputs and states of a program.

- A good test suite must include a great number of test cases.

- Good test cases always are complicated ones.

- Software test automation can replace test engineers to perform good software testing.

- Software testing is simple and easy. Anyone can do it. No training is needed.
 Topic: Introduction to Software Testing

Software Testing Limits

- Due to the testing time limit, it is impossible to achieve total confidence.

- We can never be sure the specifications are 100% correct.

- We can never be certain that a testing system (or tool) is correct.

- No testing tools can copy with every software program.

- Tester engineers never be sure that they completely understand a software product.

- We never have enough resources to perform software testing.

- We can never be certain that we achieve 100% adequate software testing.

 Topic: Introduction to Software Testing

Software Testing Process

V&V Targets

Unit test Code & Implementation

Integration Software Design

test

Validation Requirements
test

System System engineering

test
 Topic: Introduction to Software Testing

Unit Test (Component Level Test)

Unit testing: Individual components are tested independently to
ensure their quality. The focus is to uncover errors in design
and implementation, including
- data structure in a component
- program logic and program structure in a component
- component interface
- functions and operations of a component
Unit testers: developers of the components.

White-box Black-box
interface interface
input
Internal logic, data, structure input

Operations and
Functions with I/O

output
outpu
t
operation
 Topic: Introduction to Software Testing

Integration Testing

Integration test: A group of dependent components are tested together

to ensure their the quality of their integration unit.
The focus is to uncover errors in:
- Design and construction of software architecture
- Integrated functions or operations at sub-system level
- Interfaces and interactions between them
- Resource integration and/or environment integration

Integration testers: either developers and/or test engineers.

interface interface

input
Component #1 Component #2

Operations and Operations and

Functions with I/O Functions with I/O

outpu
t

operation operation
 Topic: Introduction to Software Testing

Function Validation Testing

Validation test: The integrated software is tested based on requirements

to ensure that we have a right product.
The focus is to uncover errors in:
- System input/output
- System functions and information data
- System interfaces with external parts
- User interfaces
- System behavior and performance

Validation testers: test engineers in ITG or SQA people.

User
interface System
User External interfaces
(Operations &
Functions
& Behavior)
 Topic: Introduction to Software Testing

System Testing

System test: The system software is tested as a whole. It verifies all

elements mesh properly to make sure that all system
functions and performance are achieved in the target
environment.

The focus areas are:

- System functions and performance
- System reliability and recoverability (recovery test)
- System installation (installation test)
- System behavior in the special conditions
(stress and load test)
- System user operations (acceptance test/alpha test)
- Hardware and software integration and collaboration
- Integration of external software and the system

System testers: test engineers in ITG or SQA people.

When a system is to be marketed as a software product, a testing process called

beta testing is often used.
 Topic: Introduction to Software Testing

Test Issues in Real World

Software testing is very expensive.

How to achieve test automation?????

When should we stop software testing?

Test criteria, test coverage, adequate testing.

Other software testing:

GUI Testing
Object-Oriented Software Testing
Component Testing and Component-based Software Testing
Domain-specific Feature Testing
Testing Web-based Systems
Failure Costs
 Root Causes of Failures

• Inaccurate understanding of end user requirements.

– Inability to deal with changing requirements.
• Late discovery of serious project flaws.
– For example, modules that do not fit together.
• Untrustworthy build & release process.
– Implementation team’s chaos.
 Most Common Software problems

Incorrect calculation
Incorrect data edits & ineffective data edits
Incorrect matching and merging of data
Data searches that yields incorrect results
Incorrect processing of data relationship
Incorrect coding / implementation of business rules
Inadequate software performance
Confusing or misleading data
Software usability by end users & Obsolete Software
Inconsistent processing
Unreliable results or performance
Inadequate support of business needs
Incorrect or inadequate interfaces with other systems
Inadequate performance and security controls
Incorrect file handling

12/10/23 VIT University Chennai. 27

 Objectives of testing

Executing a program with the intent of finding an error.

To check if the system meets the requirements and be executed successfully in the Intended
environment.
To check if the system is “ Fit for purpose”.
To check if the system does what it is expected to do.
A good test case is one that has a probability of finding an as yet undiscovered error.
A successful test is one that uncovers a yet undiscovered error.
A good test is not redundant.
A good test should be “best of breed”.
A good test should neither be too simple nor too complex

12/10/23 VIT University Chennai. 28

 Topic: Introduction to Software Testing

Testing Objectives

The Major Objectives of Software Testing:

- Uncover as many as errors (or bugs) as possible in a given timeline.

- Demonstrate a given software product matching its requirement

specifications.

- Validate the quality of a software testing using the minimum cost and
efforts.

- Generate high quality test cases, perform effective tests, and issue
correct and helpful problem reports.

Major goals:

uncover the errors (defects) in the software, including errors in:

- requirements from requirement analysis
- design documented in design specifications
- coding (implementation)
- system resources and system environment
- hardware problems and their interfaces to software
 Objective of a Software Tester
Find bugs as early as possible and make sure they get fixed.
To understand the application well.
Study the functionality in detail to find where the bugs are likely to occur.
Study the code to ensure that each and every line of code is tested.
Create test cases in such a way that testing is done to uncover the hidden bugs and
also ensure that the software is usable and reliable

12/10/23 VIT University Chennai. 30

 VERIFICATION & VALIDATION

Verification - typically involves reviews and meeting to evaluate documents, plans,

code, requirements, and specifications. This can be done with checklists, issues lists,
walkthroughs, and inspection meeting.

Validation - typically involves actual testing and takes place after verifications are
completed.
Validation and Verification process continue in a cycle till the software becomes defects
free.

12/10/23 VIT University Chennai. 31

 Topic: Introduction to Software Testing

Verification and Validation

Software testing is one element of a broader topic that is often referred to as
===> Verification and Validation (V&V)

Verification --> refers to the set of activities that ensure that software correctly
implements a specific function.

Validation -> refers to a different set of activities that ensure that the software
that has been built is traceable to customer requirements.

Boehm [BOE81]:

Verification: “Are we building the product right?”

Validation: “Are we building the right product?”

The definition of V&V encompasses many of SQA activities, including

formal technical reviews, quality and configuration audits
performance monitoring, different types of software testing
feasibility study and simulation
 What is Software Testing Life Cycle (STLC)?

Software Testing Life Cycle (STLC) is defined as a sequence of activities conducted to perform
Software Testing.
It consists of series of activities carried out methodologically to help certify your software
product.
 What is Entry and Exit Criteria?

Entry Criteria:Entry Criteria gives the prerequisite items that must be completed before
testing can begin.
Exit Criteria: Exit Criteria defines the items that must be completed before testing can be
concluded
You have Entry and Exit Criteria for all levels in the Software Testing Life Cycle (STLC)