Software Quality Assurance

1
 Quality Control
• Variation control (actuality/expectation,
differences, surprises)-core of quality control
• Quality types
– Quality of design (characteristics designed for an item-
ex., performance specifications)
• Requirements
• Specifications
• design
– Quality of conformance
• Implementation

2
 Quality Control Activities
• A series of inspections, reviews, and tests
used throughout the software process to
ensure each work product meets the
requirements placed upon it.
• These activities encompasses the auditing
and reporting functions of management for
quality assurance.

3
 Costs for Quality Assurance
Prevention Appraisal Failure

•Quality planning •In- and inter-process •Internal

•Formal technical inspection Rework
reviews •Equipment calibration Repair
Failure mode analysis
•Test equipment and maintenance
•Testing •External
•Training
Complaint resolution
Product return and
replacement
Help line support
Warranty work

4
 People for SQA
• Software engineers
– Apply solid technical methods and measures
– Conduct formal reviews
– Perform well-planned software testing
• SQA group
– Assists software engineers to achieve a high
quality end product

5
 SQA Group’s work
• Prepares an SQA plan
• Participates in the development of the chosen software
process
• Reviews and verifies software engineering activities
• Verifies and audits products with specifications
• Documents formally the deviations in software work and
work products
• Records any noncompliance and reports to management

6
 SQA Activities
• Static Software inspections Concerned with
analysis of the static system representation to
discover problems (static verification)
– May be supplement by tool-based document and code
analysis
• Dynamic Software testing Concerned with
exercising and observing product behaviour
(dynamic verification)
– The system is executed with test data and its
operational behaviour is observed

7
 Software Reviews (Inspections)
• Reviews are applied at various points
during software development to uncover
errors/defects that can be removed.
• Industry studies indicate that 50-65 percent
all errors happened in design activities;
formal reviews can detect up to 75 percent
of them.

8
 Cost for Error Removal
• Assume that an error uncovered during
design will cost 1 monetary unit to correct.
• Same error before testing – 6.5 units
• Same error during testing – 15 units
• Same error after release – 60-100 units

9
 Defect Amplification without
Review
Preliminary design
Detail design
0
10 6 Code/Unit test
0 0% 6
37 10 10
10 4*1.5 0%
4 94
25 27*3 20%
94 Integration Test 27
25
Validation Test
94
47 47 System test
0 50%
24 24
0 0 50% 12

0 0 50%

0
10
 Defect Amplification with
Review
Preliminary design
Detail design
0
3 2 Code/Unit test
0 70% 2
15 5 5
10 1*1.5 50%
1 24
25 10*3 60%
24 Integration Test 10
25
Validation Test
24
12 12 System test
0 50%
6 6
0 0 50% 3

0 0 50%

0
11
Software Verification and Validation Techniques

Inspection
Desk checking

Walkthrough Peer review Testing

12
 Software Inspections
• Inspector examines the representation with the aim
of detecting anomalies and defects.
• It does not require execution so it can be used
before implementation.
• It can be applied to any representation of the
system (requirements, design, code, test data, etc.)
• It is an effective technique for error detection.

13
 Program Inspection
• An approach for detecting defects, not
correcting defects.
• Defects may be logical errors, anomalies in
the code that might indicate an erroneous
condition (e.g. an uninitialized variable) or
non-compliance with standards.

14
 Program Inspection
• Step by step reading the program.
• Check against a list of criteria
– common errors
– standards
– consistency rules

15
 Inspection Pre-condition
• A precise specification must be available.
• Team members must be familiar with the
organisation standards.
• Syntactically correct code must be available.
• An error checklist should be prepared.
• Management must accept that inspection will
increase costs early in the software process.
• Management must not use inspection for
performance evaluation.
16
 Inspection Procedure
• System overview presented to inspection team.
• Code and associated documents are distributed to
inspection team in advance.
• Inspection takes place and discovered errors are
noted.
• Modifications are made to repair discovered errors.
• Re-inspection may or may not be required.

17
 Fagan Inspection Technique
• A checklist of inspection items.
• It includes design and code inspections.
• 4 - 5 inspectors: moderator, designer,
programmer, tester.
• It is good for real-time systems because
errors are not repeatable.
• Different programmers tend to have
different error patterns.
18
 Design Inspection
• Precondition: functional requirements and
design specification must exist.
• Design inspection checks for
– completeness
– consistency, and
– correctness.

19
 Walkthrough
• The developer loudly reads through the
program.
• The developer provides explanation if
he/she deems necessary.
• Other team members may ask questions and
stimulate doubts.
• The developer answers questions and
justifies answers.
20
Difference Between Walkthrough
and Inspection
Inspection
Walkthrough
• use asimple
list oftest
criteria
data
– common
• team is lederrors
through manual simulation
– standards
• check product step by step while reading
– consistency rules
aloud
•• team inspects
stimulate doubttheand
product for common
discussions
errors and compliance to standards and
consistency rules

21
 Peer Review
• The product is reviewed by peers, guided by a list
of review questions.
• The review questions are designed to assess the
product.
• The reviewers are given one to two weeks to
review the work.
• The review results may vary due to difference in
reviewer’s knowledge, experience, background,
and criticality.
• A review meeting is usually conducted to discuss
the review results. 22
 Peer Review Procedure
1. A product overview is presented to the reviewers,
who are assigned products to review.
2. The reviewers evaluate the product and answer
the review questions independently.
3. At the review meeting, the reviewers present
their comments and suggestions, and the
developer answers questions and clarifies doubt.
4. After the review meeting, the developer fixes the
problems and produces a summary of solutions.
The reviewers check the changes.
5. A second review meeting may be needed.
23
 Verification and Validation in the Life Cycle
We must ensure that Software Requirements We must check
the development Analysis and make sure that
activities are carried SRS the SRS, design,
out correctly. and code are
Software Design correct.
software design
Coding & Unit
verificatio Testing static validation
n
code
modules We must ensure that the
Integration & software executes and
Integration Testing produces expected
results.

Acceptance integrated software

Testing

Maintenance
dynamic validation &
testing
24
 Verification
• Verification is the process of checking that
the “implementation” conforms to the
“specification.”
• A lower level abstraction is the
implementation of a higher level
abstraction.
• Design verification involves checking that
the software design satisfies the
requirements specification.
25
 Validation
• Validation aimed at checking the
correspondence between a model and the
real world, e.g., a functional specification
corresponds to the real needs of the
customer.
• A process that seeks to refute a model.
• Functional testing is a validation technique.

26
 Static and Dynamic Verification
& Validation
• Review and inspection are concerned with
analysis of the static system representation
to detect problems (static verification)
• Testing is concerned with executing the
product and observing its behaviour
(dynamic verification)

27
 Formal Verification
• A precise mathematical analysis process.
• It checks the correspondence between a
specification and an implementation.
• Example:
– specification: a logical expression of some
constraint – bad things should not happen
– implementation: a state machine model
representing the design of a system
– formal verification: model checking to find a
state that violates the logical expression. 28
 Informal Verification
• The correspondence is usually checked by a
manual process.
• Techniques include desk-checking,
inspection, walkthrough, and peer review

29