Software Quality Assurance

Pavithra Subashini
Senior lecturer
Faculty of Computing
Fundamental Test Process
Fundamental Test Process
1. Planning and control

2. Test analysis and

design

3. Test implementation
and execution

4. Evaluating exit
criteria and reporting

5. Test closure
activities:
 Fundamental Test Process cont.….
1. Planning and control:

• Defining test strategy and policies are fundamental for establishing a clear roadmap and set of
rules which could be shared with the stakeholders. This is crucial for ensuring effective testing
in a timely fashion.

• Main 6 key points for identifying required planning and control activities:
– determine scope, risks and test objectives: the goal is to make sure that each requirement
is covered by a test plan.
– determine test approach in term of test procedures, test techniques, teams, environment
and data involved: this ensures to clearly identify what's required (feasibility) from coming
testing activities.
– implement test policy/strategy previously determined.
– determine test resources (environment, people) needed so they can be planned/allocated.
– schedule all test activities: plan, design, implementation, execution, evaluation.
– determine the exit criteria which could be test coverage (% expected), number of test
executed... It's a useful step which has to be agreed with stakeholders because it
determines the end of testing and therefore software/product release.
 Planning and control cont..

• Regarding controlling test activities, the objective is ensure that

what was previously planned is well implemented and
communicated. This includes the following actions:
• measure and analyze the results: how much test has been
executed, how much defects have been found classified by their
severity..
• monitor and document progress, because stakeholders, project
managers and the testing team want to be aware of what has been
performed so far, what has been identified and what's left to verify.
• initiate corrective actions as a test strategy can be tailored or
refined during progress.
• make decisions regarding stopping or restarting testing activities or
simply confirming GO for release.
 Fundamental Test Process cont.….

2. Test analysis and design:

• This is in relation with understanding software requirements
and defining test objectives and in order to allow that, we
will:
• review test basis and understand clearly the software
specifications and deliverable.
• identify test conditions, so list every feature, transaction,
function, quality attribute or other structural element that
could be verified by one or more test cases.
• evaluate testability of requirements: specify expected results
and make sure requirements can be verified.
• design test environment: identify any tool or infrastructure
required for testing the software/product.
 Fundamental Test Process cont.….
3.Test implementation and execution:

• All high-level design performed has now to be built in test cases and manual or automated test
procedures. So before execution, following preparation tasks are required:
– develop and prioritize our test cases by describing step by step instructions needed to
perform the test.
– create test suites or a logical collection of common test cases to execute for ensuring that
test scheduling is well managed and the feature/transaction clearly end-to-end tested.
– implement and verify the environment to make sure that it's ready and will not introduce
errors or block us in test execution.
• Then, during execution :
– ensure that test suites and individual test cases are run in the planned sequenced with the
right tools and environment.
– log the outcome of each test execution and record software/product identities and version
tested, data used, tools and environment involved. This is very important and contributes
to make sure that configuration items are well managed.
– compare actual results to expected results and provide additional information and details
on how the defect occurs, ensuring reproducibility.
– retest after the defect has been fixed and automatically perform regression test.
 Fundamental Test Process cont.….

4. Evaluating exit criteria and reporting

• For each test level, exit criteria has to be defined
beforehand and a reporting has to be provided to
inform stakeholders for recording official decision taken:
• checks test logs against exit criteria in test planning and
identify what's remain to test or to fix.
• assess if more tests are needed and if initial exit criteria
has to be reset and agreed again with stakeholders.
• write a summary report as a test deliverable on which
clear decision could be documented. .
 Fundamental Test Process cont.….

2. Test closure activities

• Ending testing includes various and sequential
closing activities that would lead to lesson learning.
Therefore, we have to:
• check which planned acceptance or rejection
deliverable has been delivered, defects resolved or
differed in a future release.
• finalize and archive test ware for an eventual reuse
(script, data, tools, environment).
Test Estimation Techniques…
 What is Estimation?
 Is it a Science?

 Is it art?

 A bit of both - Design...

11
 What is an Estimate ?

 An estimate is a forecast or prediction.

 An Estimate is an approximation of what it would Cost.
A Rough Idea how long a Task would take to
 complete.
An Approximate calculation of Quantity or Degree or
 Worth.

12
 How to measure - Estimate

 An Estimate is Effort required in terms of

 Time
 Cost
 An Estimate is based on
 Past Experience/ Past Data
 Available Documents/ Knowledge
 Assumptions
 Calculated Risks

13
 Why do we Estimate?
 Overshooting Budgets Exceeding Timescales
 Example of Overshooting Budgets and Timescales :

 A Standish Group survey of 8,000 software projects found that
the average project exceeded its planned budget by 90 percent
and its schedule by 120 percent.
 Several industry studies have reported that fewer than half of
software projects finish within their allotted schedules and
budgets.

14
 Class activity

• Identify importance of having

proper estimation.

Time -10 minutes

Individual task
 Importance of Estimates Cont…
A study conducted by KPMG Pete Marwick found these causes of
software runaways:
Project Objectives Not Fully Specified (51%)

Bad Planning and Estimating (48%)

Technology New to the Organization (45%)
Inadequate/No Project Management Methodology (42%)
Insufficient Senior Staff on the Team (42%)
Poor Performance by Suppliers of Hardware/Software (42%)

© Nishant Worah 2009 16

 Estimation for Testing
Questions to be asked
Are the Requirements finalized?

If not then, How frequently the requirements change?

What are the Types of Testing required?

Are Test Assets already available?

If Test Assets are available, What Stage they are in ?

Is the Infrastructure for Testing ready?

Have the Assumptions/Risks been documented.

Are the Responsibilities clear.

17
 Steps involved in Estimation
Identify scope

State assumptions

Assess the tasks involved

Estimate the effort for each task

Calculate Total Effort

Work out elapsed time / critical path

Finish and Yes No Reassess tasks

Present the Is the total
and / or effort
Estimate reasonable
18
 Test Estimation Process
Step 1 - Define Function Point

Step 2 - Give Weightage

to all Function Points

Step 3 - Define an
Estimate Per
Function Point

Step 4 - Calculate Total

Effort Estimate
19
Function Point Method
 What is a Function Point ?
•  Function points are a measure of the size of computer
applications and the projects that build them.
• The size is measured from a functional, or user,

point of view.
• It is independent of the computer language,
 development methodology, technology or

capability of the project team used to develop the

application.
 Function Point can be based upon Use Cases
20

 Function Point Method – Step 1

Define Function Points

 Function Point is measured
from a functional, or user, point
of view.
It is independent of computer
language,
 development
methodology, technology or
capability of the project team.
Function Point Method – Step 2
Give Weightage to all Function
Points
 Complex
 API/ Low level Interactions
 Complex Batch
 Processes
Medium
 Database Checks Simple
 Batch Processes
 Interaction between two or
functionalities
 Simple
 GUI
 Involving only One Functionality
22
Function Point Method – Step 3

Define an Estimate Per Function Point

 Based on similar
Projects Experience.
Organization/Project


Standards.

23
Function Point Method – Step 4

Total Effort Estimate =

Total Function Points *

Estimate defined Per

Function Point

24
 Function Point Method - Example
Function
Weightage Points Total

Simple 1 35
35
Medium 3 20 60
Complex 5 5 25

Function Point Total 120

Estimate defined Per Function Point 4.15

Total Estimated Effort( Person Hours) 498

25
 To Conclude - Few Tips
• Allow enough time to do a proper Project Estimate.
• Use documented data from similar past Projects.
• Use Own Estimates.
• Use several different people to Estimate.
• Re-Estimate the Project several times throughout its
lifecycle.
• Create a standardized estimation procedure.
• Focus some effort on improving organization’s software
Project estimation
process.

26
Summary

 Define the scope

 State your assumptions
 Document your estimate
 Assess if estimate is reasonable
 Communicate - with requester
and PM
 Check actual effort against
estimate
 Use feedback next time

27