Unit-IV Basics of Software Testing and Automation Testing

(Weightage- 10 , Hrs- 08)

4.1 Software Testing-Objectives, Error and bug terminology
Software testing
Definition:
The process or method of finding error/s in a software application or program so that
the application functions according to the end user's requirement is called software testing.
Objectives of Software testing

• Finding defects which may get created by the programmer while developing the
software.

• Gaining confidence in and providing information about the level of quality.

• To prevent defects.

• To make sure that the end result meets the business and user requirements.

• To ensure that it satisfies the SRS that is System Requirement Specifications.

• To gain the confidence of the customers by providing them a quality product.

Benefits of Software Testing

• Cost-Effective: Testing any IT project on time helps you to save your money for the
long term.

• Security: People are looking for trusted products. It helps in removing risks and
problems earlier.

• Product quality: It is an essential requirement of any software product. Testing

ensures a quality product is delivered to customers.

• Customer Satisfaction: The main aim of any product is to give satisfaction to their
customers.
Bug, Fault & Failure

• Error : An error is a human action that produces the incorrect result that results in a
fault.

• Bug : The presence of error at the time of execution of the software.

• Fault : State of software caused by an error.

• Failure : Deviation of the software from its expected result. It is an event.

 • A person makes an Error That creates a fault in software That can cause a failure in
operation.

4.2 The Fundamental Test Process

Testing is a process rather than a single activity. This process starts from test planning then
designing test cases, preparing for execution and evaluating status till the test closure. So,
we can divide the activities within the fundamental test process into the following basic
steps:

STLC Phases

There are following six major phases in every Software Testing Life Cycle Model (STLC
Model):

1. Requirement Analysis
2. Test Planning
3. Test case development
4. Test Environment setup
5. Test Execution
6. Test Cycle closure

What is Entry and Exit Criteria in STLC?

 Entry Criteria: Entry Criteria gives the prerequisite items that must be completed
before testing can begin.
 Exit Criteria: Exit Criteria defines the items that must be completed before testing
can be concluded.
Requirement Phase Testing
Requirement Phase Testing also known as Requirement Analysis in which test team studies
the requirements from a testing point of view to identify testable requirements and the QA
team may interact with various stakeholders to understand requirements in detail.
Requirements could be either functional or non-functional. Automation feasibility for the
testing project is also done in this stage.
Activities in Requirement Phase Testing

 Identify types of tests to be performed.

 Gather details about testing priorities and focus.
 Prepare Requirement Traceability Matrix (RTM).
 Identify test environment details where testing is supposed to be carried out.
 Automation feasibility analysis (if required).
Deliverables of Requirement Phase Testing

 RTM
 Automation feasibility report. (if applicable)
 Test Planning in STLC

Test Planning in STLC is a phase in which a Senior QA manager determines the test plan
strategy along with efforts and cost estimates for the project. Moreover, the resources, test
environment, test limitations and the testing schedule are also determined. The Test Plan
gets prepared and finalized in the same phase.

Test Planning Activities

 Preparation of test plan/strategy document for various types of testing

 Test tool selection
 Test effort estimation
 Resource planning and determining roles and responsibilities.
 Training requirement

Deliverables of Test Planning

 Test plan /strategy document.

 Effort estimation document.

Test Case Development Phase

The Test Case Development Phase involves the creation, verification and rework of test
cases & test scripts after the test plan is ready. Initially, the Test data is identified then
created and reviewed and then reworked based on the preconditions. Then the QA team
starts the development process of test cases for individual units.
Test Case Development Activities

 Create test cases, automation scripts (if applicable)

 Review and baseline test cases and scripts
 Create test data (If Test Environment is available)

Deliverables of Test Case Development

 Test cases/scripts
 Test data

Test Environment Setup

Test Environment Setup decides the software and hardware conditions under which a work
product is tested. It is one of the critical aspects of the testing process and can be done in
parallel with the Test Case Development Phase. Test team may not be involved in this
activity if the development team provides the test environment. The test team is required to
do a readiness check (smoke testing) of the given environment.
Test Environment Setup Activities

 Understand the required architecture, environment set-up and prepare hardware

and software requirement list for the Test Environment.
 Setup test Environment and test data
 Perform smoke test on the build

Deliverables of Test Environment Setup

 Environment ready with test data set up

 Smoke Test Results.

Test Execution Phase

Test Execution Phase is carried out by the testers in which testing of the software build is
done based on test plans and test cases prepared. The process consists of test script
execution, test script maintenance and bug reporting. If bugs are reported then it is
reverted back to development team for correction and retesting will be performed.
Test Execution Activities

 Execute tests as per plan

 Document test results, and log defects for failed cases
 Map defects to test cases in RTM
 Retest the Defect fixes
 Track the defects to closure

Deliverables of Test Execution

 Completed RTM with the execution status

 Test cases updated with results
 Defect reports

Test Cycle Closure

Test Cycle Closure phase is completion of test execution which involves several activities
like test completion reporting, collection of test completion matrices and test results.
Testing team members meet, discuss and analyze testing artifacts to identify strategies that
have to be implemented in future, taking lessons from current test cycle. The idea is to
remove process bottlenecks for future test cycles.
Test Cycle Closure Activities

 Evaluate cycle completion criteria based on Time, Test coverage, Cost,Software,

Critical Business Objectives, Quality
 Prepare test metrics based on the above parameters.
 Document the learning out of the project
  Prepare Test closure report
 Qualitative and quantitative reporting of quality of the work product to the
customer.
 Test result analysis to find out the defect distribution by type and severity.

Deliverables of Test Cycle Closure

 Test Closure report

 Test metrics
4.3 Quality Assurance and Quality Control, Testing, Verification and Validation.

• Quality is meeting the requirement, expectation, and needs of the customer is free
from the defects.

• QA is focuses on preventing defect.

• QC is focuses on identifying a defect.

Verification and Validation
Verification:

• Verification is a static analysis technique.

• Testing is done without executing the code.

• Examples – Reviews, Inspection, and walkthrough.

Validation:

• Validation is a dynamic analysis technique,

• where testing is done by executing the code.

• Examples - functional & non-functional testing techniques.

4.4 Automation Testing : Introduction, Features of test tool, Guideline for selecting a tool

What is Automation Testing?

Automation Testing or Test Automation is a software testing technique that performs
using special automated testing software tools to execute a test case suite.
Manual Testing is performed by a human sitting in front of a computer carefully executing
the test steps.
The automation testing software can also enter test data into the System Under Test,
compare expected and actual results and generate detailed test reports.

Software Test Automation demands considerable investments of money and resources.

Why Test Automation?

Test Automation is the best way to increase the effectiveness, test coverage, and
execution speed in software testing. Automated software testing is important due to the
following reasons:

 Manual Testing of all workflows, all fields, all negative scenarios is time and money
consuming
 It is difficult to test for multilingual sites manually
 Test Automation in software testing does not require Human intervention. You can
run automated test unattended (overnight)
 Test Automation increases the speed of test execution
 Automation helps increase Test Coverage
 Manual Testing can become boring and hence error-prone.

Automated Testing Process:

Following steps are followed in an Automation Process

Step 1) Test Tool Selection

Step 2) Define scope of Automation

Step 3) Planning, Design and Development

Step 4) Test Execution

Step 5) Maintenance
Test Tool selection largely depends on the technology the Application Under Test is built
on

Define the scope of Automation

The scope of automation is the area of your Application Under Test which will be
automated. Following points help determine scope:

 The features that are important for the business

 Scenarios which have a large amount of data
 Common functionalities across applications
 Technical feasibility
 The extent to which business components are reused
 The complexity of test cases
 Ability to use the same test cases for cross-browser testing

Planning, Design, and Development

During this phase, you create an Automation strategy & plan, which contains the following
details-

 Automation tools selected

 Framework design and its features
 In-Scope and Out-of-scope items of automation
 Automation testbed preparation
 Schedule and Timeline of scripting and execution
 Deliverables of Automation Testing

Test Execution
Automation Scripts are executed during this phase. The scripts need input test data before
there are set to run. Once executed they provide detailed test reports.

Execution can be performed using the automation tool directly or through the Test
Management tool which will invoke the automation tool.

Example: Quality center is the Test Management tool which in turn it will invoke QTP for
execution of automation scripts. Scripts can be executed in a single machine or a group of
machines. The execution can be done during the night, to save time.

Test Automation Maintenance Approach

Test Automation Maintenance Approach is an automation testing phase carried out to
test whether the new functionalities added to the software are working fine or not.
Maintenance in automation testing is executed when new automation scripts are added
and need to be reviewed and maintained in order to improve the effectiveness of
automation scripts with each successive release cycle.
 Static and dynamic testing tool, Advantages and Disadvantages of using tools,
When to use Automated test tools.

Static testing tool Dynamic testing tool

These tools are used by developers as

These tools require the code to be
part of the development and component
in a “running state”
testing process

code is not executed or run but tool itself is

They analyse rather than testing
executed

testing is of two types like

1) Static testing
2) Dynamic testing;
accordingly the tools used during these testing are also known as
1) Static testing tools
2) Dynamic testing tools
Static testing tools seek to support the static testing process whereas dynamic
testing tools support dynamic testing process. It may be noted that static testing is
different from dynamic testing.
Few points of differences among static and dynamic testing are as under:

Static Testing Dynamic Testing

Static testing does not require the Dynamic testing involves testing the
1 actual execution of software. software by actually executing it.
2 It is more cost effective. It is less cost effective.
It achieves less than 50% statement coverage
It may achieve 100% statement because it finds bugs only in part of codes
3 coverage in relatively short time. those are actually executed.
It may involve running several test cases,
each of which may take longer then
4 It usually takes shorter time. compilation.
It uncovers limited type of bugs that are
5 It may uncover variety of bugs. explorable through execution.
It can take place only after executables are
6 It can be done before compilation. ready
Software testing tools are frequently used to ensure consistency, thoroughness and
efficiency in testing software products and to fulfil the requirements of planned
testing activities. These tools may facilitate unit (module) testing and subsequent
integration testing (e.g., drivers and stubs) as well as commercial software testing
tools.

Testing tools can be classified into following two categories:

Static Test Tools: These tools do not involve actual input and output. Rather, they
take a symbolic approach to testing, i.e. they do not test the actual execution of the
software. These tools include the following: ,
1) Flow analyzers: They ensure consistency in data flow from input to
output.
2) Path tests: They find unused code and code with contradictions.
3) Coverage analyzers: It ensures that all logic paths are tested.
4) Interface analyzers: It examines the effects of passing variables and
data between modules.
Dynamic Test Tools: These tools test the software system with ‘live’ data. Dynamic
test tools include the following
1) Test driver: It inputs data into a module-under-test (MUT).
2) Test beds: It simultaneously displays source code along with the program
under execution.
3) Emulators: The response facilities are used to emulate parts of the system
not yet developed.
4) Mutation analyzers: The errors are deliberately ‘fed’ into the code in order
to test fault tolerance of the system.

When Should You Automate Your Software Testing?

1. When the cost makes sense.
2. When you are using repetitive tests.
3. When time will be saved.
4. When quality is sure to be improved.
5. When tests are run frequently.
6. When you need to run multiple tests at once.
7. Get started with automation.