PROCESS ASSESSMENT AND IMPROVEMENT

ADVANCED SOFTWARE ENGINEERING

WEEK – 17

DR. KAUSHALYA DISSANAYAKE

PhD in Computer Science (MSU, MALAYSIA),
MSc in Cyber Security (SLIIT, SL),
PGD in IT (SLIIT, SL),
BSc (UOR, SL)
SOFTWARE PROCESS ASSESSMENT & IMPROVEMENT I

• The existence of a software process is no guarantee that software will be delivered on

time, that it will meet the customer’s needs, or that it will exhibit the technical
characteristics that will lead to long-term quality characteristics.

• Process patterns must be coupled with solid software engineering practice.

• The process itself should be assessed to ensure that it meets a set of basic process
criteria that have been shown to be essential for a successful software engineering.
SOFTWARE PROCESS ASSESSMENT & IMPROVEMENT II

• The figure shows the relationship between the software process and the methods
applied for assessment and improvement.
• ISO/IEC 15504:1998. Information technology – Software process assessment.

Process

Is
examined Identifies
Identifies by capability
changes to and risks of
Process
Assessment

leads leads
to to
Process Capability
Improvement motivates Determination
IMPROVEMENT PROCESS IN ISO 12207

The Improvement Process is a process for establishing, assessing, measuring,

controlling, and improving a software life cycle process.
This process consists of three following activities:
1) Process establishment;
2) Process assessment;
3) Process improvement.
IMPROVEMENT PROCESS: PROCESS ESTABLISHMENT

This activity consists of the following task:

▪ The organisation shall establish a suite of organisational processes for all software
life cycle processes as they apply to its business activities.
▪ The processes and their application to specific cases shall be documented in
organisation’s publications.
▪ As appropriate, a process control mechanism should be established to develop,
monitor, control, and improve the process(es).
IMPROVEMENT PROCESS: PROCESS ASSESSMENT

This activity consists of two following tasks.

Process Assessment Procedure

▪ A process assessment procedure should be developed, documented, and applied.
▪ Assessment records should be kept and maintained.

Process Reviews
▪ The organisation shall plan and carry out reviews of the process at appropriate
intervals to ensure its continuing suitability and effectiveness in the light of
assessment results.
IMPROVEMENT PROCESS: PROCESS IMPROVEMENT

Improvement
• The organisation shall effect improvements to its processes as it determines to be
necessary as a result of process assessment and review.
• Process documentation should be updated to reflect improvement in the
organisational processes.
IMPROVEMENT PROCESS: PROCESS IMPROVEMENT

Historical, Technical and Evaluation Data Collection

• Historical, technical and evaluation data should be collected and analysed to gain an
understanding of the strengths and weaknesses of the employed processes.
• These analyses should be used as feedback to improve these processes, to
recommend changes in the direction of the projects for subsequent projects, and to
determine technology advancement needs.
IMPROVEMENT PROCESS: PROCESS IMPROVEMENT

Quality Cost Data Collection

• Quality cost data should be collected, maintained, and used to improve the
organisation’s processes as a management activity.
• These data serve to purpose of establishing the cost of both the prevention and
resolution of problems and non-conformity in software products and services.
THE CMMI CONCEPT: AN OVERVIEW OF CMMI

The CMMI Product Suite contains an enormous amount of information and guidance to
help an organization improve its processes.
1. Materials to help evaluate the content of processes – information that is essential
to technical, support, and managerial activities.
2. Materials to help improve process performance – information that is used to
increase the capability of the organization’s activities.
AN OVERVIEW OF CMMI: PROCESS CONTENT I

CMMI provides guidance for managerial processes.

▪ You should establish and maintain a plan for managing your work, and make sure
everyone involved is committed to performing and supporting the plan.
▪ When you plan, you should establish exactly how you develop and maintain cost,
schedule, and product estimates.
▪ When you do the work that you plan, you should compare the performance and
progress to the plan and take corrective actions if you find actual and planned results
out of sync.
▪ You should establish and maintain agreements with your suppliers and make sure
that you both satisfy them.
▪ Etc.
AN OVERVIEW OF CMMI: PROCESS CONTENT II

CMMI guidance on technical matters includes ways to develop, elaborate, and manage
requirements, and to develop technical solutions that meet those requirements.
▪ It includes ways to develop technical solutions that meet those requirements.
▪ It reminds you that the integration of product components depends on good
interface information, and it needs to be planned and verified.
▪ You should make sure that the products and services you develop are consistent
with the requirements and satisfy the customer’s needs through verification and
validations practices.
AN OVERVIEW OF CMMI: PROCESS IMPROVEMENT I

Processes need to be planned, and it helps if the organization has given some weight and
validity to it through policy.
▪ You need to make sure that resources are available for trained, empowered people
to perform the process.
▪ Work products and the process documentation should be controlled, and the
progress against the process plan tracked.
▪ There should be someone responsible for objectively evaluating that the process is
being followed, and management should be briefed periodically on the process
performance.
CMMI OBJECTIVES I

• The CMMI project had both initial and longer-term objectives.

• The initial objective (represented in version 1.1 of the CMMI Product Suite) was to
integrate three specific process-improvement models: software, system engineering,
and integrated product development.
CMMI OBJECTIVES II

• The longest-term objective was to lay a foundation for the later addition of other
disciplines (such as supplier sourcing, manufacturing, acquisition, or safety and
security) to CMMI.
• The figure on the next slide illustrates these objectives and the product line
approach developed by the CMMI Team.
CMMI OBJECTIVES III
CMMI OBJECTIVES IV

• CMMI-SW – the CMMI models that contains the discipline of Software.

• CMMI-SE/SW – the model that contains the disciplines of Systems Engineering and
Software.
• CMMI-SE/SW/IPPD – the model that adds materials for Integrated Product and
Process Development to CMMI-SE/SW.
• CMMI-SE/SW/IPPD/SS: Extending the name with /SS indicates inclusion of the
Supplier Sourcing material.
CMMI CONTENT: PROCESS AREAS I

• The fundamental organizational feature of all CMMI models is the “process area”.
• Not everything related to processes and process improvement is included in a
process-improvement model.
• CMMI selects only the most important topics for process improvement and then
groups those topics into “areas”.
PROCESS AREAS II

• This classification results in CMMI version 1.1 models with a relatively small set of
process areas:
 ▪ 22 in CMMI-SE/SW and CMMI-SW,
 ▪ 24 in CMMI-SE/SW/IPPD, and
 ▪ 25 in CMMI-SE/SW/IPPD/SS.

• We will discuss the CMMI process areas in more detail below.

PROCESS AREAS

• In addition to this purpose, the process area has goals that describe the results of a
successful requirements management process and practices that can help achieve
these goals.
• Purpose - the reason for which something is done.
• Goal – what should be reached.
• Practice – how it may be reached.
CMMI CONTENT: CONTENT CLASSIFICATION I

• In the CMMI models, a distinction is drawn between the terms “required”,

“expected”, and “informative”.
The most important material is required;
• these items are essential to the model and to an understanding of what is needed for
process improvement and demonstrations of conformance to the model.
CMMI CONTENT: CONTENT CLASSIFICATION II

Second in importance if the expected material;

• this items may not be fully essential, and in some instances they may be present in
an organization that successfully uses the model.
Third in importance (and largest in quantity) is informative material;
• these items constitute the majority of the model. The informative materials provide
useful guidance for process improvement.
CMMI CONTENT: REQUIRED MATERIALS I

• The sole required component of the CMMI models is the “goal”.

• A goal represents a desirable end state, the achievement of which indicates that a
certain degree of project and process control has been achieved.
• When a goal is unique to a single process area, it is called a “specific goal” (SG).
• In contrast, when a goal may apply across all the process areas, it is called a “generic
goal” (GG).
REQUIRED MATERIALS II

• Each process area has between one and four specific goal; the entire
CMMI-SE/SW/IPPD/SS model (version 1.1) includes a total of 55 specific goals.

Process Area Specific Goal

Requirements REQM SG 1: Requirements are managed and
Management inconsistencies with project plans and work product
are identified.
Project PMC SG 2: Corrective actions are managed to
Monitoring closure when the project’s performance or results
and Control deviate significantly from the plan.
REQUIRED MATERIALS III

• In contrast to a specific goal, a generic goal has a scope that crosses all of the
process areas.
• Because their application is so broad, the wording of a generic goal tends to be
somewhat more abstract than the wording of a specific goal (see an example on the
next slide).
REQUIRED MATERIALS IV

CMMI Glossary CMMI Definitions

Terms
Managed process A “managed process” is a performed process
that is planned and executed in accordance
with policy; employs skilled people having
adequate resources to produce controlled
output; involves relevant stakeholders; is
monitored, controlled, and reviewed; and is
evaluated for adherence to its process
description.
CMMI CONTENT: EXPECTED MATERIALS: PRACTICE

• The only expected component of the CMMI models is the statement of a “practice”.
• A practice represents the “expected” means of achieving a goal. Every practice in the
CMMI models is mapped to exactly one goal.
SPECIFIC AND GENERIC PRACTICES

• While CMMI practices (as a class) are not required, they act as strong indicators that
the goals to which they are mapped are being met.
• When a practice is unique to a single process area, it is called a “specific practice”
(SP).
• When a practice may apply across all the process areas, it is called a “generic
practice” (GP).
SPECIFIC PRACTICE I

The entire CMMI-SE/SW/IPPD/SS version 1.1 model includes a total of 189 specific
practices, which are mapped to the 55 specific goals.

Specific Goal Specific Practice

REQM SG 1: Requirements are REQM SP 1.1-1: Develop an
managed and inconsistencies understanding with the
with project plans and work requirements providers on the
products are identified. meaning of the requirements.
SPECIFIC PRACTICE II

• When the specific practices mapped to a specific goal are taken as a unit, they
provide additional insight into how the goal should be understood.
• The table on the next slide shows one goal together with its full complement of
practices.
SPECIFIC PRACTICE III

Specific Goal Specific Practices

Configuration CM SP 1.1-1: Identify the configuration
Management (CM) items, components, and related work
SG 1: Baselines of products that will be placed under
identified work configuration management.
products are CM SP 1.2-1: Establish and maintain a
established. configuration management and change
management system for controlling work
products.
CM SP 1.3:1: Create or release baseline for
internal use and for delivery to the
customer.
GENERIC PRACTICE

• For example, one generic practice that is mapped to the generic goal to
institutionalise a managed process (GG 2) addresses the training of people.
 GP 2.5: “Train the people performing or supporting the process as need.”
CMMI CONTENT: INFORMATIVE MATERIALS

CMMI models contain 10 types of informative components.

Read yourselves about them.
CMMI REPRESENTATIONS I

• An organization may choose to approach process improvement from the perspective

of either process area capability or organizational maturity.
• This approach is supported in a continuous representation, with one key term
“capability”.
CMMI REPRESENTATIONS II

• In contrast, an organizational maturity perspective emphasizes sets of process areas

that are intended to define proven stages of process maturity across the organization.
• This approach is employed in a staged representation, with its use of the key term
“maturity”.
• The CMMI models (version 1.1) for each combination of disciplines are presented with
two representations, continuous and staged.
STAGED MODELS I

• A staged model provides a predefined road map for organizational improvement

based on proven grouping and ordering of processes and associated organizational
relationships.
• The term “staged” comes from the way that the model describes this road map as a
series of “stages” that are called “maturity levels”.
• Each maturity level has a set of process areas that indicate where an organization
should focus to improve its organizational process.
STAGED MODELS II

The CMM for Software is the primary example of a staged model.

The figure shows the CMM for Software and its five stages.
STAGED MODELS III

The table shows the key process areas within Levels 1 and 2.
Level Focus Key Process Areas
Level 2: Basic project Software Configuration Management
Repeatable management Software Quality Assurance
Software Acquisition Management
Software Project Control
Software Project Planning
Requirements Management
Level 1: Competent None
Initial people and
heroics
STAGED MODELS IV

• The key process areas at level 2 of the CMM for Software focus on the software
project’s concerns related to establishing basic project management controls.
• Level 3 addresses both project and organizational issues, as the organization
establishes an infrastructure that institutionalises effective software engineering and
management processes across all projects.
• The key process areas at level 4 focuses on establishing a quantitative understanding
of both software process and the software work products being built.
• Level 5 covers the issues that both the organization and the projects must address
to implement continual, measurable software process improvement.
CONTINUOUS MODELS I

• Continuous models provide less specific guidance on the order in which

improvement should be accomplished.
• They are called continuous because no discrete stages are associated with
organizational maturity.
• Like the staged models, continuous models have process areas that contain
practices.
CONTINUOUS MODELS II

• Unlike in staged models, however, the practices of a process area in a continuous

model are organized in a manner that supports individual process area growth and
improvement.
• Most of the practices associated with process improvement are generic; they are
external to the individual process areas and apply to all process areas.
CONTINUOUS MODELS III

• The results can be reported as a capability profile, like that shown in the figure.
• The capability profile may include just the level number ratings, as in the figure, or it
may also contain detailed information at the goal or practice levels.
CMMI MODEL REPRESENTATIONS

• The CMMI Team was given an unprecedented challenge: Create a single model that
could be viewed from distinct perspectives, continuous and staged.
SELECTION OF PROCESS AREAS IN CMMI

• A major challenge centred on the selection of the process areas for CMMI.
• A second issue was whether the process areas in the two representations would
need to be (or should be) identical.
ORGANIZATION OF PROCESS AREAS I

• The 25 process areas of CMMI-SE/SW/IPPD/SS (Version 1.1) are divided into four
maturity levels in the staged representation and four process categories in the
continuous representation.
• The following tables show these two structures, respectively.
ORGANIZATION OF PROCESS AREAS II

Table: Staged Groupings

Staged Grouping Acronyms Process Areas

Maturity level 2 REQM Requirements Management
PP Project Planning
PMC Project Monitoring and Control
SAM Supplier Agreement Management
MA Measurement and Analysis
PPQA Process and Product Quality Assurance
CM Configuration Management
ORGANIZATION OF PROCESS AREAS III
Staged Grouping Acronyms Process Areas
Maturity level 3 RD Requirements Development
TS Technical Solution
PI Product Integration
VER Verification
VAL Validation
OPF Organization Process Focus
OPD Organization Process Definition
OT Organizational Training
IPM Integrated Project Management
RSKM Risk Management
IT Integrated Teaming
ISM Integrated Supplier Management
DAR Decision Analysis and Resolution
OEI Organizational Environment for Integration
ORGANIZATION OF PROCESS AREAS IV

Staged Acronyms Process Areas

Grouping
Maturity level 4 OPP Organization Process Performance
QPM Quantitative Project Management
Maturity level 5 OID Organizational Innovation and
CAR Deployment
ORGANIZATION OF PROCESS AREAS V

Table: Continuous Groupings

Continuous Acronyms Process Areas

Grouping
Process OPF Organizational Process Focus
Management OPD Organizational Process Definition
OT Organizational Training
OPP Organizational Process Performance
OID Organizational Innovation and
Deployment
ORGANIZATION OF PROCESS AREAS VI

Continuous Acronyms Process Areas

Grouping
Project PP Project Planning
Management PMC Project Monitoring and Control
SAM Supplier Agreement Management
IPM Integrated Project Management
RSKM Risk Management
IT Integrated Teaming
ISM Integrated Supplier Management
QPM Quantitative Project Management
ORGANIZATION OF PROCESS AREAS VII

Continuous Acronyms Process Areas

Grouping
Engineering REQM Requirements Management
RD Requirements Development
TS Technical Solution
PI Product Integration
VER Verification
VAL Validation
ORGANIZATION OF PROCESS AREAS VIII

Continuous Acronyms Process Areas

Grouping
Support CM Configuration Management
PPQA Process and Product Quality Assurance
MA Measurement and Analysis
DAR Decision Analysis and Resolution
OEI Organization Environment for
Integration
CAR Causal Analysis and Resolution
CAPABILITY DIMENSION I

• CMMI includes six levels of process area capability, imaginatively called capability
levels (CLs), and numbered 0 through 5.
• These CLs indicate how well the organization performs in an individual process area.
• Capability level 0 (CL 0) indicated that the processes are NOT Performed, which means
that one or more of the specific goals of the process area are not satisfied.
• The capability levels increase up to capability level 5 (CL 5), where the process is
performed well and is being continuously improved.
CAPABILITY DIMENSION II
Capability Generic Goal
Level
CL 0 No goal
CL 1 GG 1: The process supports and enables achievements of the
specific goals of the process area by transforming
identifiable input work products to produce
identifiable output work products.
CL 2 GG 2: The process is institutionalised as a managed process.
CL 3 GG 3: The process is institutionalised as a defined process.
CL 4 GG 3: The process is institutionalised as a quantitatively
managed process.
CL 5 GG 3: The process is institutionalised as an optimising
process.
CAPABILITY DIMENSION III
CAPABILITY DIMENSION IV
MATURITY DIMENSION I

• The maturity dimension of a CMMI model is described in the staged representation.

• Five levels of maturity exist, each of which indicates the process maturity of the
organization.
MATURITY DIMENSION II
MATURITY DIMENSION III

• Generic goals and practices follow different inclusion rules in stages representation
that they do in the continuous representation.
• They are included in the process areas according to the level where the process area
is staged.
• Process area stages at maturity level 2 use the CL 2 generic goal and its associated
generic practices.
MATURITY DIMENSION
CAPABILITY LEVEL 0 GENERIC PRACTICES

• Capability level 0 does not have any generic practices.

• It is the Not Performed level, which means that the specific goals for a process area
are not satisfied.
CAPABILITY LEVEL 1 GENERIC PRACTICES

• When GP 1.1 is applied to a process area, the base specific practices of the process
area are performed.
• Most of the specific practices in the CMMI models are base practices, with the only
exceptions involving some advanced practices in the Engineering process areas.

Name Generic Practice

GP 1.1 Perform the base practices of the process to develop
Perform work products and provide services to achieve the
Base specific goals of the process area.
Practice
CAPABILITY LEVEL 2 GENERIC PRACTICES I
Name Generic Practice
GP 2.1 Establish an Establish and maintain an organizational policy for
Organizational planning and performing the process.
Structure
GP 2.2 Plan the Process Establish and maintain the plan for performing the
process.
GP 2.3 Provide Provide adequate resources for performing the
Resources process, developing the work products, and
providing the services of the process.
GP 2.4 Assign Assign responsibility and authority for performing
Responsibility the process, developing the work products, and
providing the services of the process.
GP 2.5 Train People Train the people performing or supporting the
process as needed.
CAPABILITY LEVEL 2 GENERIC PRACTICES II
Name Generic Practice
GP 2.6 Manage Place designated work products of the process
Configuration under appropriate levels of configuration
management.
GP 2.7 Identify and Identify and involve the relevant stakeholders as
Involve Relevant planned.
Stakeholders
GP 2.8 Monitor and Monitor and control the process against the plan for
Control Process performing the process and take appropriate
corrective actions.
GP 2.9 Objectively Objectively evaluate adherence of the process
Evaluate Adherence against its process description, standards, and
procedures, and address non-compliance.
GP 2.10 Review Status Review the activities, status, and results of the
with Higher-Level process with higher level management and resolve
Management issues.
CAPABILITY LEVEL 3 GENERIC PRACTICES

Name Generic Practices

GP 3.1 Establish and maintain the description of a
Establish a defined process.
Defined Process
GP 3.2 Collect Collect work products, measures, measurement
Improvement results, and improvement information derived
Information from planning and performing process to
support the future use and improvement of the
organization’s processes and process assets.
CAPABILITY LEVEL 4 GENERIC PRACTICES

Name Generic Practices

GP 4.1 Establish and maintain quantitative objectives for
Establish the process that address quality and process
Quality performance based on customer needs and
Objectives business objectives.
GP 4.2 Stabilize the performance on one or more sub-
Stabilize Sub- processes to determine the ability of the process to
process achieve the established quality and process-
Performance performance objectives.
CAPABILITY LEVEL 5 GENERIC PRACTICES

Name Generic Practices

GP 5.1 Ensure Ensure continuous improvement of the
Continuous Process process in fulfilling the relevant business
Improvement objectives of the organization.
GP 5.2 Correct Identify and correct the root causes of
Common Cause of defects and other problems in the process.
Problems
CMMI PROCESS AREAS I

• In the continuous representation, the process areas are grouped by category:

Process Management, Project Management, Engineering, and Support.
• The process areas in three of the four categories (Support is the exception) are
related to one another.
• The Support process areas are independent of each other as well as all other
process areas.
• In the staged representation, the process areas are grouped into maturity levels 2
through 5.
CMMI PROCESS AREAS II

Category Process Area Maturity

Level
Process Organizational Process Definition 3
Management Organizational Process Focus 3
Organizational Training 3
Organizational Process Performance 4
Organizational Innovation and 5
Deployment
CMMI PROCESS AREAS III

Category Process Area Maturity

Level
Project Project Planning 2
Management Project Monitoring and Control 2
Supplier Agreement Management 2
Integrated Project Management for IPPD 3
Risk Management 3
Integrated Teaming 3
Integrated Supplier Management 3
Quantitative Project Management 4
CMMI PROCESS AREAS V

Category Process Area Maturity

Level
Engineering Requirements Management 2
Requirements Development 3
Technical Solution 3
Product Integration 3
Verification 3
Validation 3
CMMI PROCESS AREAS VI

Category Process Area Maturity

Level
Support Configuration Management 2
Process and Product Quality Assurance 2
Measurement and Analysis 2
Decision Analysis and Resolution 3
Organizational Environment for Integration 3
Causal Analysis and Resolution 5