Configuration Verification and
Auditing
CSE3025Y
Why Configuration Verification and
Auditing?
Software Testing and Reviews are not as
comprehensive and thorough
Alpha testing:
When system has a lot of new, previously untested features
Conducted with limited users
Primarily to evaluate the success/failure (or acceptance) of
the new features
Beta testing:
When development team decides that customer evaluation
is needed before final release of product
Uncover bugs/faults in the system
Done on a much larger scale than alpha testing
Why Configuration Verification and
Auditing?
Existing testing mechanisms not sufficient,
comprehensive to provide assurance that a
product is built according to specification and
is complete in all respects
Thus, configuration audits
Such audits also provide objective evidence of
compliance of products and processes with
standards, guidelines and procedures
Introduction
Configuration audits are performed after
software integration and testing
It is a check to verify that the product package
contains all the required components and
performs as expected
Purpose and Benefits
Ensure that the product design provides agreed
performance capabilities
Validate integrity of configuration documentation
Verify consistency between product and
configuration documentation
Provide confidence in establishing a product
baseline
Provide a known configuration as a basis for
operation, maintenance and training
Resources and Materials required for
auditing
Audit plan and agenda
Applicable specifications, drawings, manuals,
schedules, test results, inspection reports
Tools and inspection equipment necessary for
evaluation and verification
Access to the product(s)
Configuration Auditing
Functional configuration auditing
Physical configuration auditing
Functional Configuration Audits
FCA is an audit conducted to verify that:
the software actually performs in accordance with
the requirements and as stated in the
documentation
Check whether the development of a Configuration
Item (CI) has been completed satisfactorily
Check whether the item has achieved the performance
and functional characteristics
Check whether the operational and support documents
are complete and satisfactory
How is the FCA conducted?
Test plans, test data and test methodology are
reviewed
To verify that all functional parameters were
tested
To verify that any change to the CI are in line with
the specification requirement
To verify that there are no unintended
consequences as a result of change
May include various forms of tests: reliability
testing, environmental tests, stress testing,
interfaces with other systems
Physical Configuration Audit (PCA)
PCA is conducted after the FCA
Aims:
To verify that all components to be delivered
actually exist and they are complete
To verify that a built CI conforms to the technical
documentation
Demonstrate that the actual software system that will
be delivered contains the functional and physical
characteristics
To verify that software product specification and
version description documents are consistent with the
software product
Physical Configuration Audit
Audit team examines the design
documentation, source code, user
documentation and any other items that
accompany the final software system
When PCA is successfully completed, a
product baseline is established
Configuration Audits and SCM tools
SCM tools automatically capture all SCM
related information comprehensively as the
activities occur
E.g. Journal reports created by the tools record all
events that happened to the CIs, this creates an
audit trail which can be used by auditors
Querying facilities offered by the tools allow
auditors to obtain any required information for
the auditing process
Review Questions
I. Evaluate the Rationale for Configuration
Auditing for (i) a new product (ii) a new version
of an existing product?[5 MARKS]
II. Describe what is meant by a Product Baseline
and what are the prerequisites (in terms of
auditing) for establishing a product baseline?
[5+5 MARKS]
III. Explain, what is the role of the SCM team in
configuration audits? [5 MARKS]
IV. Compare and Contrast: Configuration Auditing
and Auditing of the SCM system [5 MARKS]
