OVERVIEW:
FUNDAMENTALS OF SOFTWARE
TESTING
By Dung Nguyen
Date: Sep. 23, 2009
Duration: 4 hours
Course Code: GCS-C119
Contents
Introduction
Concepts: Black Box And White Box Testing
PART 1
TESTING LIFECYCLE IN WATERFALL AND RUP
The Lifecycle Of Testing in Software Development Process
(SDP)
PART 2
OVERVIEW PHASES OF TEST AND TEST TYPES
Concepts: Test Phases and Test Types
PART 3
THE RELATED ACTIVITIES IN PER TEST PHASES
Unit Test Phase
Integration Test Phase
System Test Phase
Acceptance Test Phase
Glossary
Sep. 23, 2002
Confidential
Introduction
The test process defines 4 phases (including Test Types) of
software testing which occur between product coding and product
delivery.
It also covers, in brief, pre-test activities on which the test effort
depends. This document is intended to provide an overview about
Fundamentals of Software Testing.
Sep. 23, 2002
Confidential
CONCEPT: BLACK BOX TESTING
Black box testing:
Not based on any knowledge of internal design or code.
Tests are based on requirements and functionalities.
Sep. 23, 2002
Confidential
CONCEPT: WHITEBOX TESTING
White box testing:
Based on knowledge of the internal logic of an application's
code. Tests are based on coverage of code statements,
branches, paths, and conditions.
Sep. 23, 2002
Confidential
PART 1:
TESTING LIFECYCLE IN WATERFALL AND
RUP
Sep. 23, 2002
Confidential
Testing Lifecycle in Waterfall
Sep. 23, 2002
Confidential
Testing Lifecycle in Iteration
Sep. 23, 2002
Confidential
PART 2
OVERVIEW PHASES OF TEST AND TEST
TYPES
Sep. 23, 2002
Confidential
CONCEPTS: PHASES OF TEST
There are essentially 4 test phases that have to be completed
between coding and product delivery:
1
Unit Test
Phase
Integration
Test Phase
System Test
Phase
4
Acceptance
Test Phase
Each test phase is structured to detect specific categories of
defects
The types of tests that are performed in each test phase
Sep. 23, 2002
Confidential
10
CONCEPTS: PHASES OF TEST (cont.)
Test Phases
Unit
Test Phase
Unit
Test
Responsibility
Project Team
Oversees the entire testing process
Primary Test
Platform
SDE Host
QA/QC/SW Team
Integration Test
Phase:
Software - Module
Software - Subsystem
Software, HardWare Deliverable Product
SDE Host
SDE Host
Target Platform
System Test Phase
QA/QC/SW Team
Target Platform
Acceptance Test
Phase
Acceptance Test Group
Target Platform
Sep. 23, 2002
Confidential
11
Unit Test Phase
1
Unit Test
Phase
Integration
Test Phase
System Test
Phase
4
Acceptance
Test Phase
NOTE
1. Feature testing is
NOT performed in this
phase of testing.
TEST TYPES
2.This phase catches
65% of all bugs- Ross
Collard.
Statement Coverage
3 . Effective unit
Basis Path Coverage
testing
is
the
Help and Error Messages
foundation of all later
phases of testing. No
File Handling
amount of integration
or system testing can
compensate
for
Sep. 23, 2002
Confidentialinadequate
12unit
Integration Test Phase
1
Unit Test
Phase
Integration
Test Phase
System Test
Phase
4
Acceptance
Test Phase
TEST TYPES
Inter-unit/module/subsystem
communication
Functional tests
Timing and synchronization tests
Reliability
Load and Performance
Sep. 23, 2002
Confidential
13
Integration Test Phase (cont.)
NOTE
1. Feature testing is NOT performed in this phase of
testing.
2. Stubs are used to replace components which are not
ready for the initial phase of this testing. Stubs are
replaced by actual components in the latter phase of
this testing.
3. Performance, load, and reliability test types are all
subsystem based that could be performed in this test
phase.
4. Integration testing is hierarchical. Units are
integrated into modules, and modules may be integrated
into other modules or subsystems. Full system
integration is the final stage of integration.
Sep. 23, 2002
Confidential
14
System Test Phase
1
Unit Test
Phase
Integration
Test Phase
System Test
Phase
4
Acceptance
Test Phase
TEST TYPES
Feature
Performance
Load
Reliability
Storage
Capacity
Systems Operations and
Maintenance
Sep. 23, 2002
Confidential
15
Acceptance Test Phase
1
Unit Test
Phase
Integration
Test
System Test
Phase
4
Acceptance
Test Phase
TEST TYPES
End-User tests
Administrator tests
Distributor tests
Support tests
Sep. 23, 2002
Confidential
16
PART 3
THE RELATED ACTIVITIES IN PER TEST
PHASES
Sep. 23, 2002
Confidential
17
UNIT TEST PHASE
Objective of the Unit Test Phase :
Effective unit testing is the foundation of all later phases of
testing. No amount of integration or system testing can
compensate for an inadequate unit test - Boris Beizer
A unit may be regarded as a single callable routine or function
(eg. A function in C). Unit test cases require the unit to be
defined in this manner.
Note:
The functionality provided by the unit is most likely NOT
visible to the end-user of the system
Sep. 23, 2002
Confidential
18
UNIT TEST RELATED ACTIVITIES
Write Detailed Design (includes Unit & Module
Level Integration Test Plans)
Detailed Design Review (includes review of
Unit & Module Level Integration Test Plans)
Code & Write Unit Test Cases
How many Unit
Test-related
activities are
ready?
Code Inspection
Unit Test Execution
Unit Test Report / Unit Handover
Design Group (Author or Programmer)
Sep. 23, 2002
Confidential
19
UNIT TEST PHASE (cont.)
Deliverables to Unit Test
A unit which meets the Unit Test Entrance Criteria
Unit test cases
Unit test case data (input and expected output)
Note:
Anynewormodifiedunitmustundergounittesting.
Sep. 23, 2002
Confidential
20
UNIT TEST PHASE (cont.)
Unit Test Entrance Criteria
The following must occur prior to the start of unit testing:
Review of all specifications (Product, and Feature)
Review of all appropriate design documents (High Level, and
Detailed Level)
Unit must have been code inspected
Reviewed unit test plan available
Sep. 23, 2002
Confidential
21
UNIT TEST PHASE (cont.)
Deliverables from Unit Test
The following items are delivered to the integration test primes:
Unit
Unit test data
Unit test results
Test report
Sep. 23, 2002
Confidential
22
UNIT TEST PHASE (cont.)
Unit Test Exit Criteria
A unit must successfully pass unit testing (100% unit test
case execution on complete inspected code with 100% of all
identified defects resolved)
Note:
Defect: A defect may be identified in the code, specifications, test
cases,ortestplans.
Sep. 23, 2002
Confidential
23
UNIT TEST PHASE (cont.)
Responsibility for the Unit Test Phase
The author or designer from the author s design group are
responsible for ensuring that this test phase is carried out
Sep. 23, 2002
Confidential
24
INTEGRATION TEST PHASE
Objective of the Integration Test Phase:
Integration testing tests modules, subsystems, and finally
the full- system to ensure distinct components of the
application still work in accordance to customer
requirements.
Note:
Thetestiteminintegrationtestingiseitheramodule,subassembly,
subsystem,orfull-system.
Sep. 23, 2002
Confidential
25
INTEGRATION TEST RELATED ACTIVITIES
Write High Level Design
High Level Design Review
Write Integration Test Plan (subsystem
and full-system) & Test Cases
How many
Integration Testrelated activities
are ready?
Integration Test Plan (subsystem and
full-system) & Test Case Review
Integration Test Execution (SW & HW)
Integration Test Report / Integrated System
Handover (System Test Acceptance
Meeting)
Design / System Test Groups
Sep. 23, 2002
Confidential
26
INTEGRATION TEST PHASE (cont.)
Integration Test Coverage
Focus on uncovering interfacing problems (e.g. Interface
problems between units within a module)
Inter-unit/module/subsystem communication
Functional tests
Timing and synchronization tests
Reliability
Load and Performance
Note:
Performance,load,andreliabilitytesttypesmaybeperformedinthe
integrationtestphaseifitisappropriatetotheprimaryfunctioningofthe
testitemorindicativeoffullsystemperformance.
Sep. 23, 2002
Confidential
27
INTEGRATION TEST PHASE (cont.)
Deliverables to Integration Test
The following items must be delivered prior to integration
testing:
Units which have successfully passed unit testing
Unit test report
Integration test cases
Integration test data
Target hardware platform (before full system integration
testing)
Sep. 23, 2002
Confidential
28
INTEGRATION TEST PHASE (cont.)
Integration Entrance Criteria
The following must occur prior to the start of integration
testing:
Before a unit may enter into integration testing, it must have
successfully completed unit testing
All defects identified in unit test must have been resolved
Unit test results must be provided for inspection
The reviewed integration test plan and test cases must also
be available
Sep. 23, 2002
Confidential
29
INTEGRATION TEST PHASE (cont.)
Deliverables from Integration Test
The following items are delivered to System Test:
Release notes
Integration test report
Integration Test Exit Criteria
Integration testing must be fully integrated on the target
hardware platform, and successfully complete before it
meets the integration test exit criteria (100% integration test
case execution, with 100% of all identified defects resolved )
Test results must be made available to System Test for
inspection
Note:
Defect:Adefectmaybeidentifiedinthecode,specifications,test
cases,ortestplans.
Sep. 23, 2002
Confidential
30
INTEGRATION TEST PHASE (cont.)
Responsibility for the Integration Test Phase
Designers and/or design primes are responsible for low level
integration testing. Design primes and the System Test
Group are responsible for higher level integration testing and
full system integration testing. In some instances, the
System Test Group may be augmented by a designated set
of designers
Sep. 23, 2002
Confidential
31
SYSTEM TEST PHASE
Objective of the System Test Phase
In this test phase, product features are fully tested as well
as system-wide capabilities as a production-like. The tests
are focused on ensuring that the system operates in conformity
with the product and feature specifications.
Sep. 23, 2002
Confidential
32
SYSTEM TEST RELATED ACTIVITIES
Participate in reviews of Product and
Feature Specifications, and High Level
Design Documents
Write System Test Plan & Test Cases
System Test Plan & Test Case Review
How many
System Test
related activities
are ready?
System Test Report / System Handover
(Verification Acceptance Meeting) Optional
System Test Execution
System Test Group
Sep. 23, 2002
Confidential
33
SYSTEM TEST PHASE (cont.)
System Test Coverage
All system features (feature testing)
Feature interactions
System performance
Capacity limits
Load and overload
Reliability
Recovery
Security
Installation
Conversion, and expansion.
Sep. 23, 2002
Confidential
34
SYSTEM TEST PHASE (cont.)
Deliverables to System Test
The following items must be delivered prior to system testing:
Release notes
Integration test report
Minimum hardware required
Test tools as identified in the system test plan.
Sep. 23, 2002
Confidential
35
SYSTEM TEST PHASE (cont.)
System Test Entrance Criteria
Test results/reports must be provided to System Test for
inspection
Sep. 23, 2002
Confidential
36
SYSTEM TEST PHASE (cont.)
Deliverables from System Test
Problem reports (PR s) if and when any problem is discovered
during testing which have not been reported before
At the conclusion of the system test phase, the distribution
media containing the successfully tested software will be
delivered to Acceptance
System test report and test results
System Test will also issue a report on the post-mortem of the
test effort for this test phase
Sep. 23, 2002
Confidential
37
SYSTEM TEST PHASE (cont.)
System Test Exit Criteria
System testing must successfully complete before it meets
the system test exit criteria (100% system test case
execution with the product quality requirements)
The system test report and test results must be made
available to the Acceptance Test group for inspection
Responsibility for the System Test Phase
The System Test Group is responsible for ensuring that this
test phase is carried out. In some instances, the System Test
Group may be augmented by a designated set of designers
Sep. 23, 2002
Confidential
38
ACCEPTANCE TEST PHASE
Objective of the Acceptance Test Phase
Acceptance testing is focused upon determining the products
readiness to enter the marketplace. Acceptance test will simulate
typical customer scenarios in order to ensure that the product
operates acceptably in that environment. Testing in this phase is
not expected to uncover any significant technical problems.
Sep. 23, 2002
Confidential
39
ACCEPTANCE TEST RELATED ACTIVITIES
Participate in reviews of Commercial,
Product and Feature Specifications
Write Acceptance Test Plan & Test Cases
Acceptance Test Plan & Test Case Review
How many
Acceptance Test
related activities
are ready?
Acceptance Test Execution
Acceptance Test Report / Verified
System Handover (to Field Trial)
Acceptance Test Groups
Sep. 23, 2002
Confidential
40
ACCEPTANCE TEST PHASE (cont.)
Acceptance Test Coverage
There are essentially four customer perspectives which provide
the direction for Product Verification:
End-user
Administrator
Distributor
Support (Product Support)
Test types:
Installation
Conversion
Upgrade
Modification
Network
Administration
Sep. 23, 2002
Confidential
41
ACCEPTANCE TEST PHASE (cont.)
Acceptance Test Coverage(cont.)
Compatibility
Support
Scenario
Documentation
On-line Help
Sep. 23, 2002
Confidential
42
ACCEPTANCE TEST PHASE (cont.)
Deliverables to Acceptance test
The following items must be delivered prior to acceptance
testing :
Distribution media containing software to be tested
Release notes providing all appropriate information on the
software release
Minimum hardware as required for the product under test
All test tools identified in the acceptance test plan
System test results/report
Sep. 23, 2002
Confidential
43
ACCEPTANCE TEST PHASE (cont.)
Acceptance Entrance Criteria
These items must be made available at the acceptance
meeting:
Product (hardware, firmware, software, and documentation)
quality must meet or exceed criteria
Workarounds and/or description must be available for all
open PR s
All system test cases must have been executed by the
System Test Group
Sep. 23, 2002
Confidential
44
ACCEPTANCE TEST PHASE (cont.)
Acceptance Entrance Criteria (cont.)
All system and designer test cases must be available for
audit by Acceptance
Test results for each test must be provided by development
for inspection. In the event of a fail, a PR number must also
be provided
Reviewed acceptance test plan must be available. All
acceptance test activity is provided in the test plan. The final
test plan and test cases are made available
Reviewed acceptance test cases must be available
Sep. 23, 2002
Confidential
45
ACCEPTANCE TEST PHASE (cont.)
Deliverables from Acceptance
Problem reports (PR s) if and when any problem is
discovered during testing which have not been reported
before
Acceptance test report and test results
Acceptance will also issue a report on the post-mortem of
the test effort for this test phase
Sep. 23, 2002
Confidential
46
ACCEPTANCE TEST PHASE (cont.)
Acceptance Exit Criteria
Acceptance will recommend a product that it tests if it has
quality and meets or exceeds customers requirements and
expectation based on the Product specifications
Responsibility for the Acceptance Test Phase
The Acceptance Group is responsible for ensuring that this
test phase is carried out
Sep. 23, 2002
Confidential
47
GLOSSARY
See the Glossary - attached file.
Sep. 23, 2002
Confidential
48
REFERENCES
Referent Documents:
Software Testing and Quality Assurance White Papers
Pointe Technology Group, Inc.
Toronto Lab Software Test Process
Northern Telecom
Rational Unified Process
Rational Software Corp.
Sep. 23, 2002
Confidential
49
Question/ answer
Sep. 23, 2002
Confidential
50
THANK YOU
Sep. 23, 2002
Confidential
51
