18ITE09 - SOFTWARE
QUALITY AND TESTING
Dr. P. Jayanthi, ASP, CSE
Syllabus - Basics of Testing:
◻ Introduction
◻ Definition
◻ Need for Testing
◻ Testing Approaches
◻ Essentials, features and principles of software Testing.
◻ Testing Environment:
Assessing Capabilities, Staff Competency, and User
Satisfaction
Creating an environment supportive of software testing
Building the software testing process
Testing Guidelines
Software Testing
Software Testing - Definition
• Execution of a work product with intent to find a
defect
• Detecting the difference between existing and
required conditions and to evaluate software
features
• As a process, designed to
– Prove that the program is error/defect free
– Establish that the software performs functions correctly
and fit for use
– Establish that all functional expectations are available
Why Testing is important?
Software Verification & Validation
Comparing a work product with Checking the outcome of developed
processes, standards & product with respect to expectation
guidelines of customers
Cost of Software Verification &
Validation
• If done first time – comes under appraisal cost
• If repeated times – comes under failure cost – for
organizations undergoing testing
• Software Testing – finding the difference between actual
behaviour and expected behaviour of an application
• Stages of ST as per SDLC
– Feasibility testing Begins at start of project
– Contract Testing
– Requirements Testing
– Design Testing
– Coding Testing
– Acceptance Testing Continues till the end of project
Historical Perspective of Testing
• Debugging Oriented Testing
– Developers tests – not documented
• Demonstration Oriented Testing
– Testers - through demonstration
• Destruction Oriented Testing
– Negative way of testing – change in tester’s responsibility
• “proving that software doesnot fail at some abnormal instances”
instead of “proving that software works under normal conditions”
• Evaluation Oriented Testing
– Evaluation based on metrics/parameters
• Prevention Oriented Testing
– Followed by highly matured organizations
– To make it defect-free
Why Testing Necessary
• Understanding of customer requirements differ from person
to person
– So, to be checked at each stage of SDLC
• Developers think what they developed is always right &
satisfies user requirements (Blind fold )
– but not correct - it should be assessed.
• Different entities – involved in different phases
– may be gaps between requirements, design, coding
• May have integration issues
– unit, integration testing
• Blind fold thoughts of developers
– So, testers should challenge them
Approaches to Testing
◻ Approaches
Big Bang Approach
TQM Approach
◻ TQM as against Big Bang Approach
◻ Characteristics of Big Bang Approach
◻ TQM in Cost Perspective
1. Big Bang Approach
◻ Testing after development completed
◻ System testing/final testing – done before releasing
the software
◻ Last part of SDLC as per waterfall model
◻ Final product tested- before release
◻ Has main thrust on black box testing
1. Big Bang Approach
• Phase-wise defect distribution
Development Phase Percentage of Defects
Requirements 58
Design 35
Coding 5
Other 2
• Characteristics
– All defects found in last phase
– Cost-huge
– Huge rework, retesting, scrap, sorting of software components
– No corrective & preventive actions – arise from these defects
– Regression testing – correcting defects leads to creation of new defects –
dependent components affected
1. Big Bang Approach
• Organizations following Big Bang Approach
– Are less matured
– May spend huge cost of failure
– Needs several retesting & Regression Testing
– Observables:
• Verification activities (during SDLC) – can find 2/3 of total
defects
• Validation activities (in unit testing) – can find ¾ of remaining
defects
• Verification activities (during System testing) – 10% of total
defects
• Remaining 5% to 10% defects – identified only when the product
reaches customer – unless the organization takes preventive
measures
doesn’t
Big Bang Preventive Measures
go for
2. TQM Approach
• TQM Cost Triangle
– Stages of maturity
• 0 – highly matured
– No need of verification/validation
– Quality is free
• 1 – Verification done
– Will have appraisal cost
• 2 – validation
– filters the defects escaped from verification
– High validation cost
• 3 – defects found by customer
– Results in 1000 times much higher cost
2. TQM Approach
◻ Less defects produced
◻ Very few defects undetected
◻ Defect removal costs
After coding (approx. 10 times >) before coding
During Production (approx. 100 times>) before coding
◻ TQM – aims at reducing cost of development, cost
of quality by continual improvement
TQM Cost Perspective
• Green Money / Cost of Prevention
– Money spent on defining process, training people, developing quality
– Investment in doing quality work
– Improves profitability
• Blue Money / Cost of Appraisal
– Cost incurred in development, 1st time review/testing
– Doesn’t return profit
– Includes all appraisal techniques used during SDLC such as 1st time
verification/validation
• Red Money / Cost of Failure
– Pure loss for organization
– Money lost in rework, sorting, scrap
– Directly reduces profit
– Customer doesn’t pay for it
– Includes cost incurred in re-inspection, re-testing, regression testing
Views of Testing
◻ Manager’s View:
Product must be safe, reliable, when used by users
Satisfy user requirements
Testing process- should uncover defects & give
confidence to customer
◻ Tester’s View:
To discover defects, faults, weakness of product
◻ Customer’s View:
Testing must give confidence
All defects removed
s/w must be defect-free by testing
Objectives of Testing
◻ Must find
What should be done & What should not be done
Deviation from requirement specification
Risks- what should be done
◻ Software Testing
A SQA activity – in many places
But in Reality - it is a Quality Control (QC) activity
Basic Principles of Testing
◻ Define the expected output or result for each test
case
◻ Defects may be in the product or test case or test
plan
◻ Developers or the development teams must not test
their own products
◻ Inspect the results of each test completely and
carefully
◻ Used to identify the weak processes, build right
processes and improve the process capability
Basic Principles of Testing
◻ Include test cases for invalid and unexpected
conditions
◻ Test the program to see if it does what it is not
supposed to do and what it is supposed to do
◻ Do not plan tests with an assumption that no error
will be found
◻ The probability of locating more errors in any
module is directly proportional to the no. of errors
already found in that module
Successful Testers & Test Cases
◻ Successful Testers – must conduct SWOT analysis
of software and the processes used to make it
◻ Successful Test Case – must consider all cases -
+ve, -ve, all scenarios
Testing during SDLC
◻ Requirement Testing
Mock running of future application using requirement statements
Completeness of Requirement statements
User Expectation clarity
Measurability of expected results
Testability, Traceability of requirements at the end
◻ Design Testing
completeness
◻ Code Testing
Readability, maintainability in future, ability to integration testing
◻ Test Scenario & Test Case Testing
Should be feasible
Should coverall requirements
Test Case – should cover all scenarios
Requirement Traceability Matrix
◻ Tracing from requirements to coding, design….
◻ A blueprint of software under development
◻ Matrix Structure
Requirement Traceability Matrix
◻ Advantages:
Used to understand the software in a better way
Helps in tracing if any software requirement is not
implemented (OR) any gaps between requirements
and design & code &…
Entire software can be tracked completely
Test case failure can be tracked completely
◻ Problems:
Theoretically – all software must have
In reality – most software doesnot have
WHY???????
Requirement Traceability Matrix
◻ Why???
If number of requirements is high,
■ Preparing RTM is difficult
One-one, Many – one, one- many, many - many
relationships exist
■ Needs more efforts to connect columns and rows
Requirements changes frequently
Customer doesn’t find any use of it – hence, will not
pay for it.
Requirement Traceability Matrix
◻ Horizontal Traceability
Traced from requirement ---through----test results
◻ Vertical Traceability
A requirement may have child requirements – Tracing from parent to
child requirements
◻ Bidirectional Traceability
Horizontal in both directions – requirements to results, results to
requirements RISK TRACEABILITY MATRIX
◻ Risk Traceability
Tracing the possibility of risk of failure
Essentials of Software Testing
◻ SWOT Analysis
Strengths
■ Identifying strong areas
Weakness
■ Failure possibilities
Opportunity
■ Identifying space for improvement
Threats
■ Identifying risks that results in failure
Workbench
◻ Tester’s Workbench
Workbench
◻ Tester’s Workbench
For creating test strategy
Input
Test plan
Writing test scenario Do Process
Writing test cases Check Process
Test execution
Output
Defect Management
Standards &
Retesting
Tools
Regression Testing
Rework
Important Features of Testing
Process
◻ It is a destructive process, but a constructive destruction
◻ Testing needs a sadistic approach with a consideration that
there is a defect
Testers should identify the risks to the users and test the product
accordingly
◻ If the test does not detect the defect, then it is an unsuccessful
test
◻ A test that detects the defects is a valuable investment for
development and the customer
Helps in product improvement
Identifies the weaker areas of the processes used for S/W development,
improves the processes and results in customer satisfaction
Important Features of Testing
Process
◻ It is risky to develop a software and leave it untested
before delivery
Reducing the coverage of testing is a risk associated with
the S/W
Testing must give the desired level of confidence to the
users that the product will not fail
◻ With high pressure to deliver a software as quickly as
possible, Test process must provide maximum value
in shortest time frame
Important Features of Testing
Process
◻ Highest payback comes from detecting defect early in SDLC
and preventing defect leakage/defect migration from one
phase to another
It is always economical to fix the defects as and when they appear and
conduct an analysis to find its root cause
Phase contamination – uncorrected defects in the phase where it is
detected leads to leakage of the defects to the next stage and creates
problem
◻ Organizations’ aim must be “defect prevention rather than
finding and fixing a defect”
Consider testing as a defect-prevention mechanism
Requires analysis of root cause and defect prevention mechanism to
prevent recurrence and removal of potential problems
Misconceptions about Testing
◻ Anyone can do testing, & no special skills are
required for testing
◻ Testers can test quality of product at the end of
development process
◻ Defects found in testing are blamed on developers
◻ Defects found by customers are blamed on testers
Principles of Testing
◻ Programmers/Team must avoid testing their own
work products
◻ Thoroughly inspect results of each test to find
potential improvements
◻ Initiate actions for correction, corrective action and
preventive action
Salient Features of Good Testing
◻ Capturing user requirements
Testers need to analyze and document the requirements so that they can write test scenarios and test
cases
◻ Capturing user needs
Present and future requirements, process requirements, and implicit requirements
◻ Design objectives
They state the reason for choosing a particular approach to build a S/W
Functional , UI, and performance requirements and other such requirements need to be tested
◻ User interfaces
Users ability to interact with the system, receive error messages, and act according to the instructions
Displays and reports generated by the system
◻ Internal structures
Guided by software designs and guidelines used in design and development
Ex: commenting standards to be used
◻ Execution of code
Prove that application, module and program work correctly as per the requirements
Thank
you
