ISO/IEC 15504-Evolution To An International Standard: Software Process Improvement and Practice January 2003
ISO/IEC 15504-Evolution To An International Standard: Software Process Improvement and Practice January 2003
net/publication/227668602
CITATIONS READS
21 1,129
1 author:
Terry Rout
Griffith University
80 PUBLICATIONS 551 CITATIONS
SEE PROFILE
Some of the authors of this publication are also working on these related projects:
All content following this page was uploaded by Terry Rout on 11 April 2020.
Terence P. Rout
Griffith University
Queensland 4111
Australia
Contact Details:
Terence P. Rout
School of Computing and Information Technology
Nathan Campus
Griffith University
Queensland 4111
Australia
Email: [email protected]
Phone: +61 7 3875 5046
Fax: +61 7 3875 5207
*
*
Accepted for publication in Software Process: Improvement and Practice.
Abstract
This paper describes the work currently being undertaken to progress ISO/IEC TR 15504 to
the status of a full International Standard, and outlines the changes in design that are to be
incorporated in the revision. It describes the inputs for the design decisions that were taken;
identifies the fundamental changes in the architecture of the Standard; and briefly describes
the current status of the development of the Standard.
Introduction
ISO/IEC 15504 [1] is the International Standard for Process Assessment. Its development,
with the parallel empirical studies of its use by the SPICE Project [22, 23], has spanned 10
years – the initial Study Group established by JTC1/SC7 to explore the needs and
requirements for the standard reported in 1992 [12].
The first version of the Standard was published in 1998 as a Technical Report (Type 2) [2].
This was a deliberate decision, recommended by the original Study Group report, and based
upon the JTC11 Directives [3] which state, "When the subject in question is still under
technical development or where for any other reason there is the possibility of an agreement
at some time in the future, JTC 1 may decide that the publication of a TR would be more
appropriate." At the 1998 Plenary Meeting of ISO/IEC JTC1/SC7, Working Group 10
(WG10), responsible for standards in the domain of Process Assessment, and thus for the
development and ongoing maintenance of ISO/IEC TR 15504, resolved to initiate a revision
of the document set, with the goal of preparing a revised version for full International
Standard status within the three-year period allowed for the revision of Technical Reports.
JTC1/SC7 adopted the following resolution:
JTC1/SC7 authorizes its WG10 to develop Project Requirements and Schedule for the
revision of TR15504 (Software Process Assessment). The intent is to evolve the TR into
an IS, seeking in the process as wide a representation as possible with the user
community of TR 15504. Further, JTC1/SC7 instructs WG10 to liaise with WG7 and
WG132 for this work. [4]
This paper identifies the key inputs to this revision, and sets out the design of the solution
that was determined for this revision. It concludes with a brief report on the current schedule
for publication of the Standard, and describes recommendations for transition to the new
version.
User Views
A Web-based survey of user opinions on the revision commenced in 1998. The survey was
initiated by WG10, which approved the content of the questions. The survey was promoted
widely among relevant interest groups; responses were collected through a web-based
interface hosted by the Software Quality Institute at Griffith University. The survey provided
useful insights into user opinions on the usability and usefulness of the Technical Report; of
considerable interest was the divergence of opinion on some key issues of usability.
A total of 89 responses were received (by December 2000) and analysed. The respondents
to the survey covered a wide range of industry groups and occupations; Figures 1 and 2
show an analysis of the respondents.
1
ISO (the International Organization for Standardization) and IEC (the International
Electrotechnical Commission) form the specialized system for worldwide standardization.
National bodies that are members of ISO or IEC participate in the development of International
Standards through technical committees established by the respective organization to deal
with particular fields of technical activity. ISO and IEC technical committees collaborate in
fields of mutual interest. Other international organizations, governmental and non-
governmental, in liaison with ISO and IEC, also take part in the work. In the field of information
technology, ISO and IEC have established a joint technical committee, ISO/IEC JTC 1. Sub-
Committee 7 (SC7) of JTC1 is responsible for standards in the domain of systems and
software engineering.
2
Within SC7, WG7 is responsible for standards relating to software and systems life cycle
processes; WG13 was responsible for a standard for the Software Measurement Process.
25
20
15
10
ia
er
e
el
l
um
h
t
n
n
e
ai
m
ce
en
es
al
od
es
n
iv
es
ns
in
ed
ns
rc
io
s
io
th
ac
av
et
tio
is
c
tic
ot
r
le
an
at
ea
ic
m
ct
ic
O
o
iti
e
io
tu
M
Tr
sp
ur
st
m
tra
G
tro
ef
rv
eu
rv
ru
uc
op
til
at
r
ac
es
gi
To
ro
su
to
D
Se
Se
st
er
is
ic
ac
Pe
Ed
el
Lo
uf
R
Au
Ae
In
on
in
un
ic
ev
m
an
s
an
m
bl
p;
su
,
C
an
m
ar
D
ng
ne
M
Pu
Ad
m
on
m
Ph
e
ki
ts
&a
si
ur
co
ar
C
ic
an
uc
Bu
is
d
ftw
bl
le
an
,B
Le
od
Pu
tio
Te
So
Pr
e
lth
bu
nc
ea
IT
tri
na
is
H
Fi
25
20
15
10
0
nt
er
er
r
er
er
er
er
er
er
ito
lta
ag
ne
ag
in
rit
op
ag
th
ud
ra
lW
su
O
an
gi
an
el
an
/A
/T
En
on
ev
ca
tM
D
C
or
er
on
y
y
ni
ec
lit
lit
ss
h
e
al
ch
iti
ua
ac
ua
ar
oj
on
se
is
Te
ftw
Pr
Te
Q
Q
qu
si
As
es
So
Ac
of
Pr
35
30
25
20
15
10
0
ISO 15504 CMM Trillium Bootstrap Process Other
Professional
The first component of the survey addressed the usability of the original Technical Report.
Responses were obtained from the 35 respondents with familiarity with ISO/IEC TR 15504.
The results are shown in Figure 4.
Strongly Disagree Disagree No Opinion Agree Strongly Agree
25
20
15
10
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
30
25
20
No of Responses
15
10
0
1 2 3 4 5 6
Question No
40
35
30
No of Responses
25
20
15
10
0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Question No
Design Inputs
National bodies participating in WG10 were invited to submit proposals for the revision;
discussion papers were received from the USA, Japan, and Australia. There was a general
acknowledgement of the need to reduce the overall size of the Standard; there were
significant variations in the proposals for the extent and nature of any reduction, with one
view being for the removal of all material relating to guidance on use (process improvement
and capability determination). There was also a general acknowledgement of the need to
expand the scope of the Standard beyond the software life cycle processes, following on the
emergence of ISO/IEC 15288 - Systems Life Cycle Processes [14] and ISO 18529 - Human-
centred lifecycle process descriptions [15]. The need to maintain compatibility with these
current and emerging standards was seen as of significant importance in all submissions,
and a desire to increase the flexibility of application of the Standard was evident.
At meetings held in October 1998 and March 1999, WG10 developed a detailed proposal for
the review of ISO/IEC TR 15504, including a complete statement of requirements and
documented strategy for the review [16]. The proposal was distributed throughout the
Software Engineering Standards community, and was subsequently approved by JTC1. The
defined requirements have been maintained throughout the development of the Standard; in
the latest version (29 October 1999) 27 functional and 12 non-functional requirements are
defined. Many of these re-affirm the original recommendations of the Study Group report;
there are however some significant changes of direction. The principal of these is a decision
to remove the "process dimension" of the Reference Model of ISO/IEC TR 15504-2 from the
scope of the Standard, relying on external sources for definitions of processes to be
assessed. This was seen as critical to achieving the stated wish to make the standard more
flexible and extend its scope of application. In line with this goal, the title of the Standard has
been modified – from "Software Engineering – Software Process Assessment" to
"Information Technology – Process Assessment".
Process Process
Measurement Reference
Framework Assessment
Model Model
Requirements
for Compliance
determine suitability of
3
International Standardized Profile. An ISP is an internationally agreed-to, harmonized
document which identifies a standard or group of standards, together with options and
parameters, necessary to accomplish a function or set of functions.
processes described in terms of their purpose and the outcomes resulting from
implementation.
The requirement that processes are described in terms of Process Purpose and Process
Outcomes is the critical innovation in ISO/IEC 15504. It provides for a form of definition that
is independent of implementation concerns, and that focuses on the results of process
performance. The use of this approach has been endorsed by SC7 for general use across
all of its process-oriented standards; as a result, all of these Standards are effectively
harmonised with ISO/IEC 15504. In particular, the two central life-cycle standards – ISO/IEC
12207, Software Life Cycle Processes (Amd 1) [13], and ISO/IEC 15288, Systems Life Cycle
Process [14] – formally constitute Process Reference Models for the purposes of ISO/IEC
15504.
Other forms of process model may describe sets of activities or other elements that result in
achievement of the purpose; these are outside the concern of ISO/IEC 15504. A Process
Assessment Model describes processes in terms of the evidence that may be identified that
demonstrates that the process has in fact been implemented; they generally comprise sets of
practices and descriptions of work products that serve as indicators of process performance
and process capability.
As shown in Figure 5, there can be multiple Process Assessment Models for each accepted
Process Reference Model. The requirements for conformance of Process Assessment
Models in the revised Standard are based upon the existing requirements in ISO/IEC TR
15504-2, but have been re-worked in the light of experience. Conformance is on the basis of
relationships between the Process Assessment Model and both the external Process
Reference Model and the in-built measurement framework of ISO/IEC 15504.
Process Assessment
Process Model 1a
Reference Process Assessment
Model 1 Model 1b
Conformity Requirements
Process
Reference Process Assessment
Model 3 Model 3a
(eg ISO 19529)
Measurement Framework
In the course of the revision of ISO/IEC 15504, there has been a detailed review and revision
of the definitions of the Process Attributes in the Capability Dimension. As shown in Figure 4
(above), the measurement framework remains a core element of ISO/IEC 15504, and is seen
as applicable across multiple different process domains. In the early drafts of the revision,
the whole intent and structure of the original Capability Dimension from the Technical Report
was retained, although there were several changes to increase clarity.
Following comments from the UK National Body and other sources in the course of balloting,
an Other Working Group with broad international participation was established to undertake
a full review of the Measurement Framework, with explicit attention being paid to issues of
harmonisation with ISO 9001: 2000 [19]. The general structure of the restructured
framework is given in Table 4, along with the original structure from the Technical Report. It
can be seen that the changes at the top level of the framework are limited: one of the
attributes at Level 3, and both of the attributes at Level 5, have been renamed. At a lower
level of detail, however, the changes are much more significant.
The major alterations in the details of the measurement framework are at Capability Levels 2
and 3. At Level 2, a much greater level of detail has been incorporated, and more formal
traceability to ISO 9001:2000 has been incorporated. At Level 3, although the overall
capabilities associated with the level have not changed, a different perspective on the
distribution of attributes has been adopted. The concepts for Level 4 and Level 5 capability
are generally unchanged, though the specifications have been modified to improve clarity
and understanding.
Table 5 shows the changes in the attributes at Level 2. It can be seen that the description of
Performance Management is significantly more detailed, with 6 distinct characteristics
identified, against 4 in the Technical Report. Characteristics for Work Product Management
have been clarified, removing explicit reference to "dependencies" and making terminology
consistent with usage in ISO 9001.
Level 3 attributes are shown in Table 6. In the Technical Report, the distribution of
characteristics was on the basis that one attribute addressed the "process" elements
(procedures, etc) in relation to both the definition of a standard process and its deployment,
while the other addressed the definition and deployment of resources and infrastructure. In
the new draft, PA3.1 is associated with the existence and availability of a set of standard
process assets (including resources and infrastructure), while PA3.2 addresses the
deployment of these assets as a "defined process". The set of characteristics distributed
across the two attributes are basically the same. Terminology is clarified and made more
consistent.
At Capability Level 5, the characteristics described are essentially the same; however, the
order and names of the attributes have been changed. Again, concepts have been clarified
and the linkage to statistical process control concepts made more explicit.
References
1. ISO/IEC 15504 – Information Technology – Process Assessment, Parts 1 – 5 (to be
published)
2. ISO/IEC TR 15504:1998 – Information Technology – Software Process Assessment,
Parts 1 – 9
3. ISO/IEC JTC1, Procedures for the technical work of ISO/IEC JTC 1 on Information
Technology, 2002.
4. ISO/IEC JTC1/SC7, N1939, 1998 Resolutions: Resolution 516. JTC1/SC7 N1939,
12 June 1998.
5. ISO 9000: 1994, Quality Management Systems - Guidelines for Selection and Use
6. ISO/IEC 12207: 1995, Information Technology - Software Life Cycle Processes.
7. F. Maclennan, G. Ostrolenk and M. Tobin. "Introduction to the SPICE Trials" ,in K. El
Emam, J.-N. Drouin and W. Melo (editors), SPICE - The Theory and Practice of Software
Process Improvement and Capability Determination, 1997, pp. 269-286.
8. M.C. Paulk, B. Curtis, M.B Chrissis, and C.V. Weber, Capability Maturity Model for
Software, Version 1.1. Report CMU/SEI-93-TR-24, Software Engineering Institute,
Pittsburgh, February 1993.
9. F. Coallier, and J.-N. Drouin, "Developing an Assessment Method for Telecom Software
System: an Experience Report", European Conference on Software Quality, 1992.
10. P. Kuvaja, J. Similä, L. Kranik, A. Bicego, S. Saukkonen, and G. Koch, Software Process
Assessment and Improvement: The Boostrap Approach, Blackwell, 1994.
11. Compita Ltd, Process Professional Process Portfolio, Process Professional Library
Services, 1996.
12. ISO/IEC JTC1/SC7, The Need and Requirements for a Software Process Assessment
Standard, Study Report, Issue 2.0, JTC1/SC7 N944R, 11 June 1992.
13. ISO/IEC 12207: 1995, Information Technology - Software Life Cycle Processes,
Amd 1:2000.
14. ISO/IEC 15288: 2002, Information Technology - System Life Cycle Process.
15. ISO TR 18529: 2000, Ergonomics -- Ergonomics of human-system interaction -- Human-
centred lifecycle process descriptions.
16. ISO/IEC JTC1, Proposed Modifications to the JTC 1/SC 7 Programme of Work (Revision
of ISO/IEC TR 15504: 1998), JTC1 N5848, 3 Aug 1999.
17. CMMI® Product Development Team, CMMI® -SE/SW, V1.0 Capability Maturity Model ® –
Integrated for Systems Engineering/Software Engineering, Version 1.0, SEI, 2000.
18. ISO/IEC, ISO/IEC Directives - Part 3: Rules for the structure and drafting of International
Standards, Third edition, 1997.
19. ISO 9001:2000, Quality management systems – Requirements.
20. Ho-Won Jung, Robin Hunter, Dennis R. Goldenson and Khaled El-Emam, "Findings from
Phase 2 of the SPICE Trials", Softw. Process Improve. Pract. 2001; 6: 205–242
21. ISO/IEC Guide 2, 1996, Standardization and related activities – General Vocabulary
22. Dorling, A. "SPICE: Software Process Improvement and Capability dEtermination".
Information and Software Technology, 35(6/7): 404-406, June/July 1993.
23. T.P. Rout, “The SPICE Project: Past, Present and Future”, invited keynote address,
Software Process ‘96, Brighton, December 1996.
24. Rout, T.P. and P.G. Simms, “Introduction to the SPICE Documents & Architecture”, in K.
El Emam, J-N Drouin and W. Melo: SPICE: The Theory and Practice of Software Process
Improvement and Capability Determination, IEEE Computer Society Press 1998.