Quality management

Introduction

Software quality management is concerned with ensuring that the required level
of quality is achieved in a software product. Three principal concerns:
1. At the organizational level, quality management is concerned with
establishing a framework of organizational processes and standards that
will lead to high-quality software.
2. At the project level, quality management involves the application of
specific quality processes and checking that these planned processes have
been followed.
3. At the project level, quality management is also concerned with establishing
a quality plan for a project. The quality plan should set out the quality goals
for the project and define what processes and standards are to be used.

Quality management provides an independent check on the software

development process. The quality management process checks the project
deliverables to ensure that they are consistent with organizational standards and
goals.

The quality team should be independent from the development team so

that they can take an objective view of the software. This allows them to report on
software quality without being influenced by software development issues.

A quality plan sets out the desired product qualities and how these are assessed
and defines the most significant quality attributes.
The quality plan should define the quality assessment process.
It should set out which organizational standards should be applied and, where
necessary, define new standards to be used.
Quality plans should be short, succinct documents; if they are too long, no-one will
read them.
QUALITY MANAGEMENT Page 1
 Quality plan structure:
 Product introduction;
 Product plans;
 Process descriptions;
 Quality goals;
 Risks and risk management.
Quality management is particularly important for large, complex systems. The quality
documentation is a record of progress and supports continuity of development as the
development team changes. For smaller systems, quality management needs less
documentation and should focus on establishing a quality culture. Techniques have
to evolve when agile development is used.
Software Quality
Quality, simplistically, means that a product should meet its specification. This
is problematic for software systems because there is a tension between customer
quality requirements (efficiency, reliability, etc.) and developer quality requirements
(maintainability, reusability, etc.); some quality requirements are difficult to specify
in an unambiguous way; software specifications are usually incomplete and often
inconsistent. The focus may be 'fitness for purpose' rather than specification
conformance.

Software fitness for purpose

 Has the software been properly tested?
 Is the software sufficiently dependable to be put into use?
 Is the performance of the software acceptable for normal use?
 Is the software usable?
 Is the software well-structured and understandable?
 Have programming and documentation standards been followed in the
development process?
The subjective quality of a software system is largely based on its non-functional
characteristics.
This reflects practical user experience - if the software's functionality is not what is
expected, then users will often just work around this and find other ways to do what
they want to do. However, if the software is unreliable or too slow, then it is
practically impossible for them to achieve their goals.
Software quality attributes
Safety Understandability Portability
Security Testability Usability
Reliability Adaptability Reusability
Resilience Modularity Efficiency
Robustness Complexity Learnability
QUALITY MANAGEMENT Page 2
It is not possible for any system to be optimized for all of these attributes - for
example, improving robustness may lead to loss of performance. The quality plan
should therefore define the most important quality attributes for the
software that is being developed. The plan should also include a definition of the
quality assessment process, an agreed way of assessing whether some quality, such
as maintainability or robustness, is present in the product.
The quality of a developed product is influenced by the quality of the
production process. This is important in software development as some product
quality attributes are hard to assess. However, there is a very complex and poorly
understood relationship between software processes and product quality. The
application of individual skills and experience is particularly important in software
development. External factors such as the novelty of an application or the need for
an accelerated development schedule may impair product quality.
Quality managers should aim to develop a 'quality culture' where everyone
responsible for software development is committed to achieving a high level of
product quality. They should encourage teams to take responsibility for the quality of
their work and to develop new approaches to quality improvement. They should
support people who are interested in the intangible aspects of quality and encourage
professional behavior in all team members.

Software standards
Software standards define the required attributes of a product or process. They play
an important role in quality management.
Standards may be international, national, and organizational or project standards.
Encapsulation of best practices avoids repetition of past mistakes. They are a
framework for defining what quality means in a particular setting i.e. that
organization's view of quality. They provide continuity - new staff can understand the
organization by understanding the standards that are used.
Product standards apply to the software product being developed. They include
document standards, such as the structure of requirements documents,
documentation standards, such as a standard comment header for an object class
definition, and coding standards, which define how a programming language should
be used.
Product standards may include:
 Design review form
 Requirements document structure
 Method header format
 Java programming style
 Project plan format
 Change request form
Process standards define the processes that should be followed during software
development. Process standards may include definitions of specification, design and
QUALITY MANAGEMENT Page 3
validation processes, process support tools and a description of the documents that
should be written during these processes. Process standards may include:
 Design review conduct
 Submission of new code for system building
 Version release process
 Project plan approval process
 Change control process
 Test recording process

Problems: Standards may not be seen as relevant and up-to-date by software

engineers. They often involve too much bureaucratic form filling. If they are
unsupported by software tools, tedious form filling work is often involved to maintain
the documentation associated with the standards.
Practitioners should be involved in development of standards. Engineers should
understand the rationale underlying a standard. Review standards and their usage
regularly.
Standards can quickly become outdated and this reduces their credibility amongst
practitioners. Detailed standards should have specialized tool support. Excessive
clerical work is the most
significant complaint against standards. Web-based forms are not good enough.

An international set of standards that can be used as a basis for developing quality
management systems is ISO 9001, the most general of these standards,
applies to organizations that design, develop and maintain products,
including software. The ISO 9001 standard is a framework for developing software
standards. It sets out general quality principles, describes quality processes in
general and lays out the organizational standards and procedures that should be
defined. These should be documented in an organizational quality manual.

Basics of ISO 9001 certification:

 Quality standards and procedures should be documented in an organisational
quality manual.
 An external body may certify that an organisation's quality manual conforms to
ISO 9000 standards.
 Some customers require suppliers to be ISO 9000 certified although the need
for flexibility here is increasingly recognised.
The ISO 9001 certification is inadequate because it defines quality to be the
conformance to standards. It takes no account of quality as experienced by users of
the software. For example, a company could define test coverage standards
specifying that all methods in objects must be called at least once. Unfortunately,
this standard can be met by incomplete software testing that does not include tests
with different method parameters. So long as the defined testing procedures are
followed and test records maintained, the company could be ISO 9001 certified.

QUALITY MANAGEMENT Page 4

Reviews and inspections
Reviews and inspections involve a group who examines part or all of a process or
system and its documentation to find potential problems. Software or documents
may be 'signed off' at a
review which signifies that progress to the next
development stage has been approved by
management. There are different types of review with different objectives:
 Inspections for defect removal (product);
 Reviews for progress assessment (product and process);
 Quality reviews (product and standards).

Quality reviews involve a group of people who carefully examine part or all of a
software system and its associated documentation. Code, designs, specifications,
test plans,
standards, etc. can all be reviewed.
Software or documents may be 'signed off' at a review which signifies that progress
to the next
development stage has been approved by management.

Phases in the review process:

 Pre-review activities are concerned with review planning and review
preparation.
 During the review meeting, an author of the document or program being
reviewed should 'walk through' the document with the review team.
 Post-review activities address the problems and issues that have been
raised during the review meeting.
The processes suggested for reviews assume that the review team has a face-to-face
meeting to discuss the software or documents that they are reviewing. However,
project teams are now often distributed, sometimes across countries or continents,
so it is impractical for team members to meet face to face. Remote/distributed
reviewing can be supported using shared documents where each review team
member can annotate the document with their comments.

Program inspections are peer reviews where engineers examine the source of a
system with the aim of discovering anomalies and defects. Inspections do not require
execution of a system so may be used before implementation. They may be applied
to any representation of the system (requirements, design, configuration data, test
data, etc.). They have been shown to be an effective technique for discovering
program errors.

QUALITY MANAGEMENT Page 5

Checklist of common errors should be used to drive the inspection. Error checklists
are programming language dependent and reflect the characteristic errors that are
likely to arise in the language.

Quality management and agile development

Quality management in agile development is informal rather than document-based. It
relies on establishing a quality culture, where all team members feel responsible for
software quality and take actions to ensure that quality is maintained. The agile
community is fundamentally opposed to what it sees as the bureaucratic overheads
of standards-based approaches and quality processes as embodied in ISO 9001.
Good agile practices:
 Check before check-in: Programmers are responsible for organizing their
own code reviews with other team members before the code is checked in to
the build system.
 Never break the build: Team members should not check in code that causes
the system to fail. Developers have to test their code changes against the
whole system and be confident that these work as expected.
 Fix problems when you see them: If a programmer discovers problems or
obscurities in code developed by someone else, they can fix these directly
rather than referring them back to the original developer.
The review process in agile software development is usually informal. In Scrum, there
is a review meeting after each iteration of the software has been completed (a sprint
review), where quality issues and problems may be discussed. In Extreme
Programming, pair programming ensures that code is constantly being examined and
reviewed by another team member.
Pair programming is an approach where 2 people are responsible for code
development and work together to achieve this. Code developed by an individual is
therefore constantly being examined and reviewed by another team member. Pair
programming leads to a deep knowledge of a program, as both programmers have to
understand the program in detail to continue development. This depth of knowledge
is difficult to achieve in inspection processes and pair programming can find bugs
that would not be discovered in formal inspections. Pair programming weaknesses
include:
 Mutual misunderstandings: Both members of a pair may make the same
mistake in understanding the system requirements. Discussions may reinforce
these errors.
 Pair reputation: Pairs may be reluctant to look for errors because they do not
want to slow down the progress of the project.
 Working relationships: The pair's ability to discover defects is likely to be
compromised by their close working relationship that often leads to reluctance
to criticize work partners.
When a large system is being developed for an external customer, agile
approaches to quality management with minimal documentation may be

QUALITY MANAGEMENT Page 6

impractical. If the customer is a large company, it may have its own quality
management processes and may expect the software development company to
report on progress in a way that is compatible with them. Where there are several
geographically distributed teams involved in development, perhaps from different
companies, then informal communications may be impractical. For long-lifetime
systems, the team involved in development will change. Without documentation,
new team members may find it impossible to understand development.

Software measurement
Software measurement is concerned with deriving a numeric value for an attribute
of a software product or process. This allows for objective comparisons between
techniques and processes. Although some companies have introduced measurement
programs, most organizations still make systematic use of software measurement.
There are few established standards in this area.
Software metric is any type of measurement which relates to a software system,
process or related documentation: lines of code in a program, the fog index (a code
readability test), number of person-days required to develop a component.
It allows the software and the software process to be quantified.
It may be used to predict product attributes or to control the software process.

Categories of software metrics

i) Product metrics can be used for general predictions or to identify anomalous

components. For example the size, complexity, design features, performance,

efficiency, reliability, portability. Can be further classified into;
- Dynamic metrics which are collected by measurements made of a program in
execution
- Static metrics which are collected by measurements made of the system
representations

ii) Process metrics can be used to describe the effectives and the quality of the
process that produces the software product
It may include:
 The time taken for a particular process to be completed This can be the
total time devoted to the process, calendar time, the time spent on the process
by particular engineers, and so on.
 The resources required for a particular process Resources might include
total effort in person-days, travel costs or computer resources.
 The number of occurrences of a particular event Examples of events that
might be monitored include the number of defects discovered during code
inspection, the number of requirements changes requested, the number of bug
QUALITY MANAGEMENT Page 7
 reports in a delivered system and the average number of lines of code modified
in response to a requirements change.
iii) Project metrics: describe the project characteristics and execution. Examples
are number of software developers, staffing pattern over the life cycle of the
software, cost and schedule and productivity

Software measurements can be used to:

 To assign a value to system quality attributes by measuring the
characteristics of system components, such as their cyclomatic complexity,
and then aggregating these measurements, you can assess system quality
attributes, such as maintainability.
 To identify the system components whose quality is sub-standard. For
example, you can measure components to discover those with the highest
complexity. These are most likely to contain bugs because the complexity
makes them harder to understand.
Software metrics assumptions:
 A software property can be measured accurately.
 The relationship exists between what we can
measure and what we want to know. We can only measure internal attributes
but are often more interested in external software attributes.
 The relationship has been formalized and validated.
 It may be difficult to relate what can be measured to desirable external quality
attributes.
Problems with measurement in industry:
 It is impossible to quantify the return on investment of introducing an
organizational metrics program.
 There are no standards for software metrics or standardized processes for
measurement and analysis.
 In many companies, software processes are not standardized and are poorly
defined and controlled.
 Most work on software measurement has focused on code-based metrics and
plan-driven development processes. However, more and more software is now
developed by configuring ERP systems or COTS.
 Introducing measurement adds additional overhead to processes.
Software measurement and metrics are the basis of empirical software
engineering. This is a research area in which experiments on software systems and
the collection of data about real projects has been used to form and validate
hypotheses about software engineering methods and techniques.
A quality product metric should be a predictor of product quality. Classes of
product metrics:
 Dynamic metrics which are collected by measurements made of a program in
execution;

QUALITY MANAGEMENT Page 8

  Static metrics which are collected by measurements made of the system
representations;
System components can be analyzed separately using a range of metrics. The
values of these metrics may then compared for different components and, perhaps,
with historical measurement data collected on previous projects. Anomalous
measurements, which deviate significantly from the norm, may imply that there are
problems with the quality of these components.

Software Reliability
Software reliability is another very important quality factor and is defined as
probability of failure free operation of a computer program in a specified
environment for a specified time.

For example, a program X can be estimated to have a reliability of 0.96 over 8

elapsed hours.
Software reliability can be measured, directed, and estimated using historical and
development data. The key to this measurement is the meaning of term failure.
Failure is defined as non-conformance to software requirements.

It can be graded in many different ways as shown below:

• From annoying to catastrophic
• Time to fix from minutes to months
• Ripples from fixing

It is also pertinent to understand the difference between hardware and software

reliability. Hardware reliability is predicted on failure due to wear rather than failure
due to design.
In the case of software, there is no wear and tear. The reliability of software is
determined by Mean time between failures (MTBF).

MTBF is calculated as:

MTBF = MTTF + MTTR

Where MTTF is the Mean Time to Failure and MTTR is the Mean time required to
Repair.
Arguably MTBF is far better than defects/kloc as each error does not have the same
failure rate and the user is concerned with failure and not with total error count.
A related issue is the notion of availability. It is defined as the probability that a
program is operating according to requirements at a given point in time. It can be
calculated as
Availability = (MTTF/MTBF) x 100 and clearly depends upon MTTR.

QUALITY MANAGEMENT Page 9

 Configuration Management
Software systems are constantly changing during development and
use. Configuration management (CM) is concerned with the policies, processes
and tools for managing changing software systems. You need CM because it is easy
to lose track of what changes and component versions have been incorporated into
each system version. CM is essential for team projects to control changes made by
different developers.

Configuration management activities

 Version management: Keeping track of the multiple versions of system
components and ensuring that changes made to components by different
developers do not interfere with each other.
 System building: The process of assembling program components, data and
libraries, then compiling these to create an executable system.
 Change management: Keeping track of requests for changes to the software
from customers and developers, working out the costs and impact of changes,
and deciding the changes should be implemented.
 Release management: Preparing software for external release and keeping
track of the system versions that have been released for customer use.

Fig: Configuration management activities

Agile development, where components and systems are changed several times per
day, is impossible without using CM tools. The definitive versions of components
are held in a shared project repository and developers copy these into their own
workspace. They make changes to the code then use system building tools to create
a new system on their own computer for testing. Once they are happy with the
changes made, they return the modified components to the project repository.
A development phase is where the development team is responsible for managing
the software configuration and new functionality is being added to the software.
A system testing phase is where a version of the system is released internally for
testing. No new system functionality is added. Changes made are bug fixes,
performance improvements and security vulnerability repairs. A release phase where
the software is released to customers for use. New versions of the released system
are developed to repair bugs and vulnerabilities and to include new features.

QUALITY MANAGEMENT Page 10

For large systems, there is never just one 'working' version of a system. There are
always several versions of the system at different stages of development. There may
be several teams involved in the development of different system versions.

Version management
Version management (VM) is the process of keeping track of different
versions of software components or configuration items and the systems in which
these components are used. It also involves ensuring that changes made by different
developers to these versions do not interfere with each other. Therefore version
management can be thought of as the process of managing codelines and baselines.
A codeline is a sequence of versions of source code with later versions in the
sequence derived from earlier versions. Codelines normally apply to components of
systems so that there are different versions of each component. A baseline is a
definition of a specific system. The baseline therefore specifies the component
versions that are included in the system plus a specification of the libraries used,
configuration files, etc.
Baselines may be specified using a configuration language, which allows you to
define what components are included in a version of a particular system. Baselines
are important because you often have to recreate a specific version of a complete
system. For example, a product line may be instantiated so that there are individual
system versions for different customers. You may have to recreate the version
delivered to a specific customer if, for example, that customer reports bugs in their
system that have to be repaired.

Version control (VC) systems identify, store and control access to the different
versions of components. There are two types of modern version control system.
i) Centralized systems, where there is a single master repository that
maintains all versions of the software components that are being
developed. Subversion is a widely used example of a centralized VC
system.
ii) Distributed systems, where multiple versions of the component
repository exist at the same time. Git is a widely-used example of a
distributed VC system.

QUALITY MANAGEMENT Page 11

Key features of version control systems:
 Version and release identification
 Change history recording
 Support for independent development
 Project support
 Storage management
To support independent development without interference, version control systems
use the concept of a project repository and a private workspace. The project
repository maintains the 'master' version of all components. It is used to create
baselines for system building. When modifying components, developers copy (check-
out) these from the repository into their workspace and work on these copies. When
they have finished their changes, the changed components are returned (checked-in)
to the repository. Developers check out components or directories of components
from the project repository into their private workspace and work on these copies in
their private workspace. When their changes are complete, they check-in the
components back to the repository. If several people are working on a component at
the same time, each check it out from the repository. If a component has been
checked out, the VC system warns other users wanting to check out that component
that it has been checked out by someone else.
A 'master' repository is created on a server that maintains the code produced by
the development team. Instead of checking out the files that they need, a developer
creates a clone of the project repository that is downloaded and installed on their
computer. Developers work on the files required and maintain the new versions on
their private repository on their own computer. When changes are done,
they 'commit' these changes and update their private server repository. They may
then 'push' these changes to the project repository.
Benefits of distributed version control:
 It provides a backup mechanism for the repository. If the repository is
corrupted, work can continue and the project repository can be restored from
local copies.
 It allows for off-line working so that developers can commit changes if they
do not have a network connection.
 Project support is the default way of working. Developers can compile and
test the entire system on their local machines and test the changes that they
have made.
Distributed version control is essential for open source development. Several
people may be working simultaneously on the same system without any central
coordination.
Rather than a linear sequence of versions that reflect changes to the component
over time, there may be several independent sequences resulting from branching.
This is normal in system development, where different developers work
independently on different versions of the source code and so change it in different
ways. At some stage, it may be necessary to merge codeline branches to create a

QUALITY MANAGEMENT Page 12

new version of a component that includes all changes that have been made. If the
changes made involve different parts of the code, the component versions may be
merged automatically by combining the deltas that apply to the code.

When version control systems were first developed, storage management was one
of their most important functions. Disk space was expensive and it was important to
minimize the disk space used by the different copies of components. Instead of
keeping a complete copy of each version, the system stores a list of differences
(deltas) between one version and another. By applying these to a master version
(usually the most recent version), a target version can be recreated.
As disk storage is now relatively cheap, Git uses an alternative, faster approach. Git
does not use deltas but applies a standard compression algorithm to stored files and
their associated meta-information. It does not store duplicate copies of files.
Retrieving a file simply involves decompressing it, with no need to apply a chain of
operations. Git also uses the notion of packfiles where several smaller files are
combined into an indexed single file.

Change management
Organizational needs and requirements change during the lifetime of a system, bugs
have to be repaired and systems have to adapt to changes in their environment.
Change management is intended to ensure that system evolution is a managed
process and that priority is given to the most urgent and cost-effective
changes. The change management process is concerned with analyzing the costs
and benefits of proposed changes, approving those changes that are worthwhile and
tracking which components in the system have been changed.

Factors in change analysis:

 The consequences of not making the change
 The benefits of the change
 The number of users affected by the change
 The costs of making the change
 The product release cycle
In some agile methods, customers are directly involved in change management.
The propose a change to the requirements and work with the team to assess its
impact and decide whether the change should take priority over the features planned
for the next increment of the system.

QUALITY MANAGEMENT Page 13

Release management
A system release is a version of a software system that is distributed to customers.
For mass market software, it is usually possible to identify two types of release:
major releases which deliver significant new functionality, and minor releases,
which repair bugs and fix customer problems that have been reported. For custom
software or software product lines, releases of the system may have to be produced
for each customer and individual customers may be running several different
releases of the system at the same time. As well as the executable code of the
system, a release may also include:
 Configuration files defining how the release should be configured for particular
installations;
 Data files, such as files of error messages, that are needed for successful
system operation;
 An installation program that is used to help install the system on target
hardware;
 Electronic and paper documentation describing the system;
 Packaging and associated publicity that have been designed for that release.
Factors influencing system release planning:
 Competition For mass-market software, a new system release may be
necessary because a competing product has introduced new features and
market share may be lost if these are not provided to existing customers.
 Marketing requirements The marketing department of an organization may
have made a commitment for releases to be available at a particular date.
 Platform changes You may have to create a new release of a software
application when a new version of the operating system platform is released.
 Technical quality of the system If serious system faults are reported which
affect the way in which many customers use the system, it may be necessary
to issue a fault repair release. Minor system faults may be repaired by issuing
patches (usually distributed over the Internet) that can be applied to the
current release of the system.
Release creation process:
 The executable code of the programs and all associated data files must be
identified in the version control system.
 Configuration descriptions may have to be written for different hardware and
operating systems.
 Update instructions may have to be written for customers who need to
configure their own systems.
 Scripts for the installation program may have to be written.
 Web pages have to be created describing the release, with links to system
documentation.
 When all information is available, an executable master image of the software
must be prepared and handed over for distribution to customers or sales
outlets.

QUALITY MANAGEMENT Page 14

Release tracking In the event of a problem, it may be necessary to reproduce
exactly the software that has been delivered to a particular customer. When a
system release is produced, it must be documented to ensure that it can be re-
created exactly in the future. This is particularly important for customized, long-
lifetime embedded systems, such as those that control complex machines.
Customers may use a single release of these systems for many years and may
require specific changes to a particular software system long after its original release
date.

Release reproduction To document a release, you have to record the specific

versions of the source code components that were used to create the executable
code. You must keep copies of the source code files, corresponding executable and
all data and configuration files. You should also record the versions of the operating
system, libraries, compilers and other tools used to build the software.

Release planning As well as the technical work involved in creating a release

distribution, advertising and publicity material have to be prepared and marketing
strategies put in place to convince customers to buy the new release of the system.
If releases are too frequent or require hardware upgrades, customers may not move
to the new release, especially if they have to pay for it. If system releases are too
infrequent, market share may be lost as customers move to alternative systems.

QUALITY MANAGEMENT Page 15