WHO Expert Committee on Specifications for Pharmaceutical Preparations Fifty-third report

Appendix 5
Validation of computerized systems
Background
This is a revision of the previous publication:
■ Supplementary guidelines on good manufacturing practices: validation. In:
WHO Expert Committee on Specifications for Pharmaceutical Preparations,
fortieth report. Appendix 5: Validation of computerized systems. Geneva:
World Health Organization; 2006: Annex 4 (WHO Technical Report
Series, No. 937; http://apps.who.int/medicinedocs/documents/s20108en/
s20108en.pdf).

1. Introduction and scope 161

2. Glossary 162
3. Computerized system validation protocols and reports 166
4. Supplier management 167
5. Requirements specifications 167
6. System design and configuration specifications 170
7. Design qualification 170
8. System development and project implementation 171
9. Installation qualification 172
WHO Technical Report Series, No. 1019, 2019

10. Operational qualification 172

11. Standard operating procedures and training 173
12. Performance qualification and user acceptance testing 173
13. System operation and maintenance 175
14. System retirement 178
References 178
Further reading 178

160
 Annex 3

1. Introduction and scope

1.1 Computerized systems should be validated in accordance with the
principles of quality risk management and the level of validation should be
commensurate with the identified risks, complexity and intended use. This
guide applies to systems used in good manufacturing practices (GMP) (1)
but may be extended to systems used in all good practices (GXP) activities,
as appropriate.
1.2 The purpose of validation is to confirm that the specifications of a
computerized system conform to the user’s needs and are fit for intended
use, by examination and provision of documented and objective evidence
that the particular requirements can be consistently fulfilled. Validation
should establish confidence in the accuracy, reliability and consistency in
the performance of the system, and should also ensure that all necessary
technical and procedural controls are implemented, confirming compliance
with good documentation practices for electronic data generated by the
system (1).
1.3 System elements that need to be considered in validation of a computerized
system include computer hardware and software, and related equipment,
IT infrastructure and operating system environment, and documentation
of procedures and systems, as appropriate, including user manuals. Persons
should be appropriately trained and qualified, including but not limited
to, developers, end-users, system application administrators, network
engineers, database administrators and data managers. Computerized
system validation activities should address both system functionality and
configuration, as well as any custom-developed elements.
1.4 Computerized systems should be maintained throughout the system life-
cycle, in a validated state, with risk-based controls for the operational
phase, which may include, but are not limited to, system planning;
preparation and verification of standard operating procedures (SOPs)
and training programmes; system operation and maintenance, including
handling of software and hardware updates; monitoring and review; change
management; and incident reporting, followed by system retirement.
1.5 Depending on the types of systems or typical applications, such as process
control systems (distributed control system [DCS], programmable logic
controller [PLC], supervisory control and data acquisition [SCADA]);
laboratory information management systems (LIMS); laboratory instrument
control systems; and business systems (enterprise resource planning [ERP],
manufacturing resource planning [MRP II]) used by the manufacturer.
161
 WHO Expert Committee on Specifications for Pharmaceutical Preparations Fifty-third report

Documentation covering, but not limited to, the following information and
supporting process should be accessible on-site for review:
■ purpose and scope;
■ roles and responsibilities;
■ validation approach;
■ risk management approach;
■ approved system requirement/specifications;
■ system acceptance criteria;
■ supplier selection and assessment;
■ configuration management and change-control procedures;
■ backup and recovery (application and data);
■ error handling and corrective action;
■ business continuity plan and disaster recovery;
■ maintenance and support;
■ data security, including cybersecurity;
■ validation deliverables and documentation.

2. Glossary
The definitions given below apply to the terms used in these guidelines. They
may have different meanings in other contexts.
archiving. Archiving is the process of protecting records from the
possibility of being further altered or deleted, and storing these records under
the control of independent data management personnel throughout the required
WHO Technical Report Series, No. 1019, 2019

retention period. Archived records should include, for example, associated

metadata and electronic signatures.
audit trail. The audit trail is a form of metadata that contains
information associated with actions that relate to the creation, modification or
deletion of GXP records. An audit trail provides for secure recording of life-
cycle details, such as creation, additions, deletions or alterations of information
in a record, either paper or electronic, without obscuring or overwriting the
original record. An audit trail facilitates the reconstruction of the history of
such events relating to the record, regardless of its medium, including the
“who”, “what”, “when” and “why” of the action. For example, in a paper record,
an audit trail of a change would be documented via a single-line cross-out
that allows the original entry to remain legible and documents the initials
of the person making the change, the date of the change and the reason for
162
 Annex 3

the change, as required to substantiate and justify the change. In electronic

records, secure, computer-generated, time-stamped audit trails should allow
for reconstruction of the course of events relating to the creation, modification
and deletion of electronic data. Computer-generated audit trails should retain
the original entry and document the user identification and the time/date stamp
of the action, as well as the reason for the change, as required to substantiate and
justify the action. Computer-generated audit trails may include discrete event
logs, history files, database queries or reports, or other mechanisms that display
events related to the computerized system, specific electronic records or specific
data contained within the record.
automatic or live update. A process used to bring up-to-date software
and system functionalities in a silent or announced way. More specifically, the
update takes place automatically with or without the user’s knowledge.
backup. A backup means a copy of one or more electronic files created
as an alternative in case the original data or system are lost or become unusable
(e.g. in the event of a system crash or corruption of a disk). It is important to note
that backup differs from archiving, in that backup copies of electronic records are
typically only temporarily stored for the purposes of disaster recovery and may
be periodically overwritten. Such temporary backup copies should not be relied
upon as an archiving mechanism.
business continuity plan. A documented plan that defines the ongoing
process supported by management and funded to ensure that the necessary steps
are taken to identify the impact of potential losses, maintain viable recovery
strategies and recovery plans, and assure continuity of services through personnel
training, plan testing and maintenance.
cloud based. A model for enabling on-demand network access to a
shared pool of configurable computing resources that can be rapidly provisioned
and released with minimal management effort or service provider interaction.
These computing resources should be appropriately qualified.
computerized system. A computerized system collectively controls
the performance and execution of one or more automated processes and/or
functions. It includes computer hardware, software, peripheral devices, networks
and documentation, for example, manuals and standard operating procedures,
as well as personnel interacting with hardware and software.
computerized systems validation. Confirmation by examination and
provision of objective and documented evidence that a computerized system’s
predetermined specifications conform to user needs and intended use and that
all requirements can be consistently fulfilled.
commercial off-the-shelf software (COTS). A vendor-supplied software
component of a computerized system for which the user cannot claim complete
control of the software life-cycle.
163
 WHO Expert Committee on Specifications for Pharmaceutical Preparations Fifty-third report

configuration management. A discipline applying technical and

administrative direction and surveillance to identify and document the
functional and physical characteristics of a configuration item, control changes to
those characteristics, record and report change processing and implementation
status, and verify compliance with specified requirements.
data. All original records and true copies of original records, including
source data and metadata, and all subsequent transformations and reports of
these data, which are generated or recorded at the time of the GMP activity and
allow full and complete reconstruction and evaluation of the GMP activity. Data
should be accurately recorded by permanent means at the time of the activity.
Data may be contained in paper records (such as worksheets and logbooks),
electronic records and audit trails, photographs, microfilm or microfiche, audio
or video files or any other media whereby information related to GMP activities
is recorded.
data integrity. The degree to which data are complete, consistent,
accurate, trustworthy and reliable and to which these characteristics of the data
are maintained throughout the data life-cycle. The data should be collected
and maintained in a secure manner, such that they are attributable, legible,
contemporaneously recorded, original or a true copy and accurate. Assuring data
integrity requires appropriate quality and risk management systems, including
adherence to sound scientific principles and good documentation practices (1).
data life-cycle. All phases of the process by which data are created,
recorded, processed, reviewed, analysed and reported, transferred, stored and
retrieved and monitored, until retirement and disposal. There should be a
planned approach to assessing, monitoring and managing the data and the risks
to those data, in a manner commensurate with potential impact on patient safety,
product quality and/or the reliability of the decisions made throughout all phases
of the data life-cycle.
WHO Technical Report Series, No. 1019, 2019

disaster recovery. A documented process or set of procedures to recover

and protect a business IT infrastructure in any event causing the system to be
unavailable. It appropriately defines resources and actions to be taken before,
during and after a disaster, to return the system to operational use.
functional specifications. The functional specifications define functions
and technological solutions that are specified for the computerized system, based
upon technical requirements needed to satisfy user requirements (e.g. specified
bandwidth required to meet the user requirement for anticipated system usage).
legacy system. This refers to a mature computer system, programming
language, application software, or processes that are used instead of available
upgraded versions, and that have not been qualified according to current
regulatory requirements.

164
 Annex 3

master data. A single source of business data used across multiple

systems, applications and processes and subject to change control to ensure
accuracy throughout the data life-cycle.
metadata. Metadata are data about data that provide the contextual
information required to understand those data. These include structural
and descriptive metadata. Such data describe the structure, data elements,
interrelationships and other characteristics of data. They also permit data to
be attributable to an individual. Metadata necessary to evaluate the meaning of
data should be securely linked to the data and subject to adequate review. For
example, in weighing, the number 8 is meaningless without metadata, such as,
the unit, milligram, gram, kilogram, etc. Other examples of metadata include
the time/date stamp of an activity, the operator identification (ID) of the person
who performed an activity, the instrument ID used, processing parameters,
sequence files, audit trails and other data required to understand data and
reconstruct activities.
production environment. For regulated computerized systems, the
production environment is the business and computing operating environment
in which the computerized system is being used for GMP-regulated purposes.
regression analysis and testing. A documented software verification and
validation task to determine the extent of verification and validation analysis
and testing that must be repeated when changes are made to any previously
examined software component or system.
system life-cycle. The period of time that starts when a computerized
system is conceived and ends when the system is retired and decommissioned,
taking into consideration regulatory requirements. The system life-cycle typically
includes a planning phase; a development phase that includes a design phase and
a programming and testing phase; a qualification and release phase that includes
a system integration and testing phase; a validation phase; a release phase; an
operation and maintenance phase; and, finally, a system retirement phase.
user acceptance testing. Verification of the fully configured computerized
system installed in the production environment (or in a test environment
equivalent to the production environment) to perform, as intended, in the
business process when operated by end-users trained in end-user SOPs
that define system use and control. User acceptance testing (UAT) may be a
component of the performance qualification (PQ) or a validation step separate
from the PQ.
user requirements specification. The user requirements specification
(URS), if prepared as a separate document, is a formal document that defines
the requirements for use of the computerized system in its intended production
environment.

165
 WHO Expert Committee on Specifications for Pharmaceutical Preparations Fifty-third report

3. Computerized system validation protocols and reports

3.1 A computerized system needs to be validated according to an approved
protocol and a final report including results and conclusions, prior to
routine use. All validation documentation should be appropriately retained.

Validation protocol
3.2 Validation should be executed in accordance with the validation protocol
and applicable written procedures.
3.3 A validation protocol should define the objectives and the validation
strategy, including roles and responsibilities and documentation and
activities to be performed. The protocol should at least cover the scope, risk
management approach, specification, acceptance criteria, testing, review,
personnel training and release of the computerized system for GMP use.
3.4 The validation protocol should be tailored to the system type, impact, risks
and requirements applicable to the system for which it governs validation
activities.

Validation report
3.5 A validation report should be prepared, summarizing system validation
activities.
3.6 The report should make reference to the protocol, outline the validation
process, and include an evaluation and conclusion of results. Any changes or
deviations from the validation protocol and applicable written procedures
should be described and assessed, and justification for their acceptance or
rejection should be documented. Deviations should be investigated and a
WHO Technical Report Series, No. 1019, 2019

root cause determined. A validation report should also include a summary

of procedures and training.
3.7 Test results should be recorded, reviewed, analysed and compared
against the predetermined acceptance criteria. All critical and major test
discrepancies that occurred during the verification/validation testing
should be investigated and resolved. If critical and major test discrepancies
are accepted after investigation, they should be appropriately justified.
3.8 The conclusion of the report should state whether or not the outcome of the
validation was considered successful and should make recommendations
for future monitoring where applicable. The report should be approved
after appropriately addressing any issue identified during validation, and
the system should then be released for routine GMP use.
166
 Annex 3

4. Supplier management
4.1 When third parties (e.g. suppliers, service providers) are used, such
as to provide, install, configure, validate, maintain, modify or retain a
computerized system or related service, or for data processing or system
components, including cloud-based systems, an evaluation of the supplier,
supplied system or service, and the supplier’s quality systems should be
conducted and recorded. The scope and depth of this evaluation should
be based upon risk management principles.
4.2 The competence and reliability of a supplier are key factors when selecting
a product and/or service provider. Supplier management is an ongoing
process that requires periodic assessment and review of the system or
service provided. Supplier evaluation activities may include, but are not
limited to: completion of a quality-related questionnaire by the supplier;
gathering of supplier documentation related to system development, testing
and maintenance, including supplier procedures, specifications, system
architecture diagrams, test evidence, release notes and other relevant
supplier documentation; an on-site audit of the supplier’s facilities, which
may be conducted based on risk principles to evaluate the supplier’s system
life-cycle control procedures, practices and documentation.
4.3 A contract should be in place between the manufacturer and the supplier
and/or the service provider, defining the roles and responsibilities and
quality procedures for both parties, throughout the system life-cycle. The
contract acceptor should not pass to a third party any of the work entrusted
to her/him under the contract, without the manufacturer’s prior evaluation
and approval of the arrangements.

5. Requirements specifications
5.1 Requirements specifications should be written to document user
requirements and functional or operational requirements and performance
requirements. Requirements may be documented in separate user
requirements specification (URS) and functional requirements specifications
(FRS) documents, or in a combined document.

User requirements specifications

5.2 The authorized URS document, or equivalent, should describe the intended
uses of the proposed computerized system and should define critical data
and data life-cycle controls that will assure consistent and reliable data
throughout the processes by which data are created, processed, transmitted,
167
 WHO Expert Committee on Specifications for Pharmaceutical Preparations Fifty-third report

reviewed, reported, retained and retrieved and eventually disposed. The

URS should be written in a way to ensure that the data will meet regulatory
requirements, such as the World Health Organization (WHO) Guidance on
good data and record management practices (1).
5.3 Other aspects to be included in the URS may include, but are not limited to:
■ the transaction or data to be entered, processed, reported, stored and
retrieved by the system, including any master data and other data
considered to be the most critical to system control and data output;
■ the flow of data, including that of the business process(es) in
which the system will be used, as well as the physical transfer
of the data from the system to other systems or network
components. Documentation of data flows and data process maps is
recommended, to facilitate the assessment and mitigation and control
of data integrity risks across the actual, intended data process(es);
■ networks and operating system environments that support the
data flows;
■ the system interfaces with other systems and the overall security;
■ the operating program;
■ synchronization and security controls of time/date stamps;
■ controls of both the application software as well as operating systems,
to assure system access only to authorized persons;
■ controls to ensure that data will be attributable to unique individuals
(e.g. to prohibit use of shared or generic log-in credentials);
■ controls to ensure that data related to GMP purposes is legibly
and contemporaneously recorded to durable (“permanent”) media
at the time of each step and event, and controls that enforce the
WHO Technical Report Series, No. 1019, 2019

sequencing of each step and event (e.g. controls that prevent

alteration or deletion of data in temporary memory in a manner that
would not be documented);
■ controls that assure that all steps that create, modify or delete
electronic data related to GMP purposes will be recorded in
independent, computer-generated audit trails or other metadata,
or alternate documents that record the “what” (e.g. original entry),
“who” (e.g. user ID), “when” (e.g. time/date stamp) and “why” (e.g.
reason) of the action;
■ backups and the ability to restore the system and data from backups;
■ the ability to archive and retrieve the electronic data in a manner
that assures that the archive copy preserves the full content of the
168
 Annex 3

original electronic data set, including all metadata needed to fully

reconstruct the GMP activity. The archive copy should also preserve
the meaning of the original electronic data set;
■ input/output checks, including implementation of procedures for the
review of original electronic data and metadata, such as audit trails;
■ electronic signatures;
■ alarms and flags that indicate alarm conditions and invalid
and altered data, in order to facilitate detection and a review of
these events;
■ system documentation, including system specifications documents,
user manuals and procedures for system use, data review and system
administration;
■ system capacity and volume requirements, based upon the predicted
system usage and performance requirements;
■ performance monitoring of the system;
■ controls for orderly system shutdown and recovery;
■ business continuity.
5.4 The extent and detail of the requirements should be commensurate with
the operational risk and the complexity of the computerized system. User
requirements should be specific and phrased in a way that supports their
testing or verification within the context of the computerized system.

Functional specifications
5.5 Functional specifications should describe in detail the functions,
performance and interfaces of the computerized system, based upon the
technical requirements needed to satisfy user requirements, and should be
linked to user specifications.
5.6 The functional specifications provide a basis for the system design and
configuration specifications. Functional specifications should consider
requirements for operation of the computerized system in the intended
computing environment, such as functions provided by supplier-provided
software, as well as functions required for user business processes that are
not met by commercial off-the-shelf software (COTS) functionality, and
default configurations that will require custom code development. Network
infrastructure requirements should also be taken into account. Each
described function should be verifiable.
5.7 Personnel access roles that provide the ability and/or authorization to write,
alter or access programs or configuration should be defined and qualified.
169
 WHO Expert Committee on Specifications for Pharmaceutical Preparations Fifty-third report

There should be appropriate segregation of roles between personnel

responsible for the business process and personnel for system administration
and maintenance.

6. System design and configuration specifications

6.1 System design and configuration specifications should be developed based
on user and functional requirements. Specification of design parameters
and configuration settings (separate or combined) should ensure data
integrity and compliance with the WHO Guidance on good data and record
management practices (1).
6.2 System design and configuration specifications should provide a high-
level system description, as well as an overview of the system’s physical
and logical architecture, and should map out the system business process
and relevant work flows and data flows if these have not already been
documented in other requirements specifications documents.
6.3 The system design and configuration specifications may include, as
applicable, a software design specification, in case of code development,
and configuration specifications of the software application parameters,
such as security profiles, audit trail configuration, data libraries and other
configurable elements.
6.4 In addition, the system design and configuration specifications may
also include, based upon risk, the hardware design and its configuration
specifications, as well as that of any supporting network infrastructure.
6.5 System design and configuration specifications should include secure,
WHO Technical Report Series, No. 1019, 2019

protected, independent computer-generated audit trails to track

configuration changes to critical settings in the system.

7. Design qualification
7.1 Following design qualification (DQ), a review should be conducted to
verify that the proposed design and configuration of the system is suitable
for its intended purpose and will meet all applicable user and functional
specifications.
7.2 It may include a review of supplier documentation, if applicable, and
verification that requirements specifications are traceable to proposed design
and configuration specifications. The DQ review should be documented.

170
 Annex 3

8. System development and project implementation

8.1 Once the system requirements and the system design and configuration
are specified and verified, where applicable, system development activities
may begin. The development activities may occur as a dedicated phase
following completion of specification of system requirements, design and
configuration. Alternatively, development activities may occur iteratively
as requirements are specified and verified (such as when prototyping or
rapid-development methodologies are employed).

Supplier-provided systems
8.2 For supplier-provided systems, the development controls for the supplier-
provided portion of the computerized system should be assessed during
the supplier evaluation or supplier qualification. For supplier-provided
systems that include custom components (such as custom-coded interfaces
or custom report tools) and/or require configuration (such as configuration
of security profiles in the software or configuration of the hardware within
the network infrastructure), the system should be developed under an
appropriate documented quality management system.

Custom-developed systems
8.3 For custom-developed and configurable systems, the system should be
developed under an appropriate documented quality system. For these
systems or modules, the quality management system controls should
include development of code in accordance with documented programing
standards, review of code for adherence to programing standards, and
design specifications and development testing that may include unit testing
and module/integration testing.
8.4 System prototyping and rapid, agile development methodologies may be
employed during the system build and development testing phase. There
should be an adequate level of documentation of these activities.

Preparation for the system qualification phase

8.5 The system development and build phase should be followed by the system
qualification phase. This typically consists of installation, operational and
performance testing. The actual qualification required may vary depending
on the scope of the validation project, as defined in the validation protocol
and based upon a documented and justified risk assessment.

171
 WHO Expert Committee on Specifications for Pharmaceutical Preparations Fifty-third report

8.6 Prior to the initiation of the system qualification phase, the software program
and requirements and specifications documents should be finalized and
subsequently managed under formal change control.
8.7 Persons who will be conducting the system qualification should be trained
to adhere to the following requirements for system qualification:
■ test documentation should be generated to provide evidence of testing;
■ test documentation should comply with good documentation
practices;
■ any discrepancies between actual test results and expected results
should be documented and adequately resolved, based upon risk
prior to proceeding to subsequent test phases.

9. Installation qualification
9.1 Installation qualification (IQ) – also referred to as installation verification
testing – should provide documented evidence that the computerized
system, including software and associated hardware, is installed and
configured in the intended system test and production environments,
according to written specifications.
9.2 The IQ will verify, for example, that the computer hardware on which the
software application is installed has the proper firmware and operating
system, that all components are present and in the proper condition,
and that each component is installed per the manufacturer or developer
instructions.
9.3 IQ should include verification that configurable elements of the system are
appropriately set as specified. Where appropriate, this could also be done
WHO Technical Report Series, No. 1019, 2019

during operational qualification (OQ).

10. Operational qualification

10.1 The OQ – or operational/functional verification testing – should provide
documented evidence that software and hardware function as intended
over anticipated operating ranges.
10.2 Functional testing should include, based upon risk:
■ challenges on the system’s ability to do what it should do, including
verification that significant alerts and error messages are raised
based upon alarm conditions and according to specifications;
172
 Annex 3

■ an appropriate degree of testing (such as boundary, range, limit,

and nonsense entry testing), to verify that the system appropriately
handles erroneous entries or erroneous use.

11. Standard operating procedures and training

11.1 Prior to conducting of the PQ and UAT, and prior to release of the
computerized system, there should be adequate written procedures and
documents and training programmes created defining system use and
control. These may include supplier-provided user manuals as well as
SOPs and training programmes developed in house.
11.2 Procedures and training programmes that should be developed include,
but are not necessarily limited to:
■ system use procedures that address:
– routine operation and use of the system in the intended business
process(es);
– review of the electronic data and associated metadata (such
as audit trails) and how the source electronic records will be
reconciled with printouts, if any;
– mechanisms for signing electronic data;
– system training requirements prior to being granted system access;
■ system administration procedures that address:
– granting disabling and review of user access and maintaining
security controls;
– backup/restore;
– archiving/retrieval;
– disaster recovery and business continuity;
– change management;
– incident and problem management;
– system maintenance.

12. Performance qualification and

user acceptance testing
12.1 PQ, which includes UAT, should be conducted to verify the intended
system use and administration defined in the URS and DQ, or equivalent
document.
173
 WHO Expert Committee on Specifications for Pharmaceutical Preparations Fifty-third report

12.2 The PQ should be conducted in the live environment (controls for

restricted release for GMP use may be necessary) or in a test environment
that is functionally equivalent to the live environment in terms of overall
software and hardware configuration.
12.3 PQ testing should also include, as applicable, an appropriate degree of
stress/load/volume testing, based upon the anticipated system use and
performance requirements in the production environment. Such testing
may also be performed during OQ if appropriately justified.
12.4 In addition, an appropriate degree of end-to-end or regression testing of
the system should be conducted to verify the system performs reliably
when system components are integrated in the fully configured system
deployed in the production environment.
12.5 UAT should be conducted by system users, to verify the adequacy of the
system, use of SOPs and training programmes. The UAT should include
verification of the ability to generate and process only valid data within the
computerized system, including the ability to efficiently review electronic
data and metadata, such as audit trails. SOPs should be finalized and
approved upon completion of performance qualification.

Legacy systems
12.6 The continued use of a legacy system should be justified by demonstrating
the system continues to be relevant to the GMP process being supported
and by ensuring adequate validation of the system (i.e. hardware, software,
peripheral devices, networks) has been performed.
12.7 The validation approach to be taken should aim at providing data and
information to justify and support the retrospective qualification of
WHO Technical Report Series, No. 1019, 2019

the system. It should demonstrate that the system remains in a state

of control and is fit for its intended use and, where necessary, it should
include an approach for retrospective qualification of the system with
relevant evidence.
12.8 A risk assessment should be undertaken to determine the criticality
of the system to the process or operation being supported, and a gap
analysis should identify the level of completeness of existing qualification
documentation (e.g. URS, IQ/OQ/PQ, SOPs) and state of system control,
operation and maintenance.
12.9 For legacy systems, development documentation and records appropriate
for validation may not be available. Nevertheless, the strategy should be
consistent with validation principles where assurance is established, based
174
 Annex 3

on compilation and formal review of the history of use, maintenance,

error report and change-control system records. These activities should
be based on documented URS. If historical data do not encompass the
current range of operating parameters, or if there have been significant
changes between past and current practices, then retrospective data would
not of themselves support validation of the current system.
12.10 The validation exercise should demonstrate that user requirements
and system description have been appropriately established, as well as
providing evidence that the system (i.e. hardware, software, peripheral
devices, networks, processes) has been qualified and accepted and that
GMP requirements are met.

13. System operation and maintenance

Security and access control
13.1 Manufacturers should have systems and procedures in place to ensure data
integrity and access control to computerized systems, and these measures
should be commensurate with identified risks
13.2 Suitable security measures should be in place to prevent unauthorized
entry or manipulation or deletion of data through the application software,
as well as in operating system environments in which data may be stored
or transmitted. Data should be entered or amended only by persons who
are qualified and authorized to do so.
13.3 The activity of entering data, changing or amending incorrect entries, or
creating backups should be done in accordance with SOPs.
13.4 Security should extend to devices used to store programs and data. Access
to these devices should be controlled.
13.5 Measures for protecting audit trails from alteration or unauthorized
deletion should be in place. Procedures for review of audit trails, and when
necessary metadata, should define the frequency, roles and responsibilities
and nature of these reviews.
13.6 Operation of the system and acquisition of data should be traceable and
should identify the persons who made entries and/or changes, approved
decisions or performed other critical steps in system use or control.
13.7 Details of user profiles and access rights to systems, networks, servers,
computerized systems and software should be documented and reviewed
periodically. An up-to-date list on the individual user rights for the
175
 WHO Expert Committee on Specifications for Pharmaceutical Preparations Fifty-third report

software, individual computer systems and networks should be maintained

and subjected to change control. The level of detail should be sufficient
to enable computer system validation personnel, as well as IT personnel/
any external auditor/inspector, to ascertain that security features of the
system and of software used to obtain and process critical data cannot be
circumvented.
13.8 All GMP computerized systems, either stand-alone or in a network, should
have a system that is commensurate with identified risks for monitoring
through an audit trail of events that are relevant. These events should
include all elements that need to be monitored to ensure that the integrity
(1) of the data could not have been compromised without leaving a trace,
such as, but not limited to, changes in or deletion of data; changes in dates,
times, backups, archives or user access rights; and addition/deletion of
users and log-ins, in accordance with WHO Guidance on good data and
record management practices (1). The configuration and archiving of these
audit trails should be documented and also be subjected to change control.
These audit trails should be system generated, accurate, consistent, secure,
available and convertible to a generally intelligible form throughout the
retention period, and their generation appropriately qualified.

Operation and maintenance

13.9 Entry of GMP-related data into a computerized system should be verified
by an independent authorized person and locked before release for
routine use.
13.10 Validated computerized systems should be maintained in a validated state
once released to the GMP production environment.
WHO Technical Report Series, No. 1019, 2019

13.11 There should be written procedures governing system operation and

maintenance, including, for example:
■ performance monitoring;
■ change management and configuration management;
■ problem/incident management;
■ program and data security;
■ program and data backup/restore and archiving/retrieval;
■ system administration and maintenance;
■ data flow and data life-cycle;
■ system use and review of electronic data and metadata (such as
audit trails);
176
 Annex 3

■ personnel training;
■ disaster recovery and business continuity;
■ availability of replacement parts and technical support;
■ periodic re-evaluation.
13.12 Automatic or live updates should be subject to review prior to becoming
effective.

Data migration
13.13 Where electronic data are transferred from one system to another, it
should be demonstrated that data are not altered during the migration
process. Conversion of data to a different format should be considered as
data migration. Where data are transferred to another medium, they must
be verified as an exact copy, prior to any destruction of the original data.
13.14 Procedures for data migration may vary greatly in complexity, and
measures to ensure appropriate transfer of data should be commensurate
with identified risks. Migrated data should remain usable and should
retain their content and meaning. The value and/or meaning of and links
between a system audit trail and electronic signatures should be ensured
in a migration process.

Periodic review
13.15 Computerized systems should be periodically reviewed to determine
whether the system remains in a validated state or whether there is a
need for revalidation. The scope and extent of the revalidation should be
determined using a risk-based approach. The review should at least cover:
■ system performance and functionality;
■ security;
■ maintenance;
■ review of changes including upgrades;
■ review of deviations;
■ review of incidents/events (including review of audit trail);
■ systems documentation;
■ procedures;
■ training;
■ effectiveness of corrective and preventive action.

177
 WHO Expert Committee on Specifications for Pharmaceutical Preparations Fifty-third report

13.16 Corrective and preventive action should be taken where indicated as a

result of the periodic review.

14. System retirement

14.1 System retirement should be considered as a system life-cycle phase. It
should be planned, risk based and documented. If migration or archiving
of GMP-relevant data (1, 2) is necessary, the process must be documented.
14.2 Once the computerized system or components are no longer needed,
the system or components should be retired and decommissioned, in
accordance with established authorized procedures, including a change-
control procedure and a formal plan for retirement.
14.3 Records should be archived in a readable form and in a manner that
preserves the accessibility, readability and integrity of the data of the source
electronic records throughout the required records retention period.
14.4 The outcome of the retirement activities, including traceability of the data
and computerized systems, as well as the ability to retrieve the data, should
be tested and documented in a report.

References
1. Guidance on good data and record management practices. In: WHO Expert Committee on
Specifications for Pharmaceutical Preparations, fiftieth report. Geneva: World Health Organization;
2016: Annex 5 (WHO Technical Report Series, No. 966; https://www.who.int/medicines/
publications/pharmprep/WHO_TRS_996_annex05.pdf, accessed 8 February 2019).
2. WHO good manufacturing practices for pharmaceutical products: main principles. In: WHO Expert
Committee on Specifications for Pharmaceutical Preparations, forty-eighth report. Geneva: World
WHO Technical Report Series, No. 1019, 2019

Health Organization; 2014: Annex 2 (WHO Technical Report Series, No. 986; http://academy.gmp-
compliance.org/guidemgr/files/TRS986ANNEX2.PDF, accessed 10 February 2019).

Further reading
■ Supplementary guidelines on good manufacturing practices: validation. In: WHO Expert
Committee on Specifications for Pharmaceutical Preparations, fortieth report. Geneva: World
Health Organization; 2006: Annex 4 (WHO Technical Report Series, No. 937; http://apps.who.int/
medicinedocs/documents/s20108en/s20108en.pdf, accessed 9 February 2019).
■ Appendix 6: Qualification of systems and equipment. In: WHO Expert Committee on Specifications
for Pharmaceutical Preparations, fortieth report. Geneva: World Health Organization; 2006:
Annex 4 (WHO Technical Report Series, No. 937; http://apps.who.int/medicinedocs/documents/
s20108en/s20108en.pdf, accessed 9 February 2019; (for update see Annex 3 Appendix 6 in
TRS 1019).

178
 Annex 3

■ WHO good manufacturing practices: water for pharmaceutical use. In: WHO Expert Committee
on Specifications for Pharmaceutical Preparations, forty-sixth report. Geneva: World Health
Organization; 2012: Annex 2 (WHO Technical Report Series, No. 970; Geneva, World Health
Organization 2012 (WHO Technical Report Series, No. 970; http://apps.who.int/medicinedocs/
documents/s19464en/s19464en.pdf, accessed 11 February 2019).
■ OECD series on principles of good laboratory practice and compliance monitoring. No. 17.
Advisory document of the Working Group on Good Laboratory Practice (GLP). Application of
GLP principles to computerised systems. Paris: Organization for Economic Co-Operation and
Development; 2016 (ENV/JM/MONO(2016)13; http://www.oecd.org/officialdocuments/publicdis
playdocumentpdf/?cote=env/jm/mono(2016)13&doclanguage=en, accessed 11 February 2019).
■ Annex 11: Computerised systems. In: EudraLex. The rules governing medicinal products in the
European Union. Volume 4: Good manufacturing practice (GMP) guidelines: Annex 11. Brussels:
European Commission; 2011 (SANCO/C8/AM/sl/ares(2010)1064599; https://ec.europa.eu/health/
sites/health/files/files/eudralex/vol-4/annex11_01-2011_en.pdf, accessed 11 February 2019).
■ Drug Information Association. Computerized systems used in non-clinical safety assessment:
current concepts in validation and compliance. Horsham (, PA): Drug Information Association;
(DIA), 22008.
■ GAMP® 5. A risk-based approach to compliant GxP computerized systems. Tampa (FL):
International Society for Pharmaceutical Engineering (ISPE); 2008.
■ GAMP® good practice guide. A risk-based approach to GxP compliant laboratory computerized
systems, 2nd ed. Tampa (FL): International Society for Pharmaceutical Engineering (ISPE); 2012.
■ GAMP® good practice guide. A risk-based approach to operation of GxP computerized systems
– a companion volume to GAMP 5. Tampa (FL): International Society for Pharmaceutical
Engineering (ISPE); 2010.
■ GAMP® good practice guide. A risk-based approach to regulated mobile applications. Tampa
(FL): International Society for Pharmaceutical Engineering (ISPE); 2014.
■ GAMP® good practice guide. A risk-based approach to testing of GxP Systems, 2nd edition. Tampa
(FL): International Society for Pharmaceutical Engineering (ISPE); 2012.
■ GAMP® good practice guide. Global information systems control and compliance, 2nd ed.
Tampa (FL): International Society for Pharmaceutical Engineering (ISPE); 2017.
■ GAMP® good practice guide. IT infrastructure control and compliance. Tampa (FL): International
Society for Pharmaceutical Engineering (ISPE); 2005.
■ GAMP® good practice guide. Manufacturing execution systems – a strategic and program
management approach. Tampa (FL): International Society for Pharmaceutical Engineering (ISPE);
2010.
■ National Institute of Standards and Technology, US Department of Commerce. NIST Cloud
Computing Program (NCCP) (https://www.nist.gov/programs-projects/nist-cloud-computing-
program-nccp, accessed 11 February 2019).
■ Official Medicines Control Laboratories Network of the Council of Europe. Quality management
document. Validation of computerised systems. Core document. Brussels: Council of Europe; 2018
(PA/PH/OMCL (08) 69 3R; https://www.edqm.eu/sites/default/files/medias/fichiers/Validation_
of_Computerised_Systems_Core_Document.pdf, accessed 11 February 2019).
■ Official Medicines Control Laboratories Network of the Council of Europe. Quality management
document. Validation of computerised systems. Annex 1: Validation of computerised calculation
systems. Example of validation of in-house software. Brussels: Council of Europe; 2009 (PA/
PH/OMCL (08) 87 2R; https://www.edqm.eu/medias/fichiers/NEW_Annex_1_Validation_of_
computerised_calculation.pdf, accessed 11 February 2019).
179
 WHO Expert Committee on Specifications for Pharmaceutical Preparations Fifty-third report

■ Official Medicines Control Laboratories Network of the Council of Europe. Quality management
document. Validation of computerised systems. Annex 2: Validation of databases (DB), laboratory
information management systems (LIMS) and electronic laboratory notebooks (ELN). Brussels:
Council of Europe; 2009 (PA/PH/OMCL (08) 88 R; https://www.edqm.eu/medias/fichiers/NEW_
Annex_2_Validation_of_Databases_DB_Laboratory_.pdf, accessed 11 February 2019).
■ Official Medicines Control Laboratories Network of the Council of Europe. Quality management
document. Validation of computerised systems. Annex 3: Validation of computers as part of
test equipment. Brussels: Council of Europe; 2009 (PA/PH/OMCL (08) 89 R; https://www.edqm.
eu/medias/fichiers/NEW_Annex_3_Validation_of_computers_as_part_of_tes.pdf, accessed 11
February 2019).
■ Official Medicines Control Laboratories Network of the Council of Europe. Quality management
document. Validation of computerised systems. Annex 12:
■ FDA US Food and drug Administration. CFR Code of Federal Regulations Title 21. Part 11:
Electronic records; electronic signatures (https://www.accessdata.fda.gov/scripts/cdrh/cfdocs/
cfcfr/CFRSearch.cfm?CFRPart=11, accessed 11 February 2019).
■ Guide to good manufacturing practice for medicinal products. Annexes. Annex 11 (Computerised
systems). Geneva: Pharmaceutical Inspection Co-operation Scheme; 2018.
■ Good practices for computerised systems in regulated GXP environments. Geneva: Pharmaceutical
Inspection Co-operation Scheme; 2007.
WHO Technical Report Series, No. 1019, 2019

180