Every corporation wants information technology (IT) systems that are robust enough to
meet their current requirements and flexible enough to adapt to their changing business
needs. However, most corporations have very poorly designed IT applications; thus, when
these companies need to implement a major enhancement to their existing systems or build
new applications their probability for project failure is exceedingly high: 65% - 80%
according to most surveys. This failure rate is surprising as the average company spends
approximately 5.3% of its annual revenue on IT related activities.
The executives of these same companies are trying to understand why their major IT
initiatives fail at such a high rate and why seemingly simple changes to their applications
are so costly and time consuming. These executives wanted a technique for comparing their
IT development methods with those of other firms. These desires have helped fuel the
popularity of the Systems Engineering Capability Maturity ModelSM (CMM). This article
marks the first in a series on the CMM. In these articles, I will explain what the CMM is,
what each of the six CMM levels mean, apply the CMM to data warehousing along with
metrics for each level, and illustrate why a meta data repository is the most vital application
for any company looking to move up the CMM levels.
What is the Capability Maturity Model?
There are many different applications of the Capability Maturity Models. Some of these
models target software development, staffing, etc. In this series I will focus on the Systems
Engineering Capability Maturity ModelSM (CMM). The CMM is service marked by the
Software Engineering Institute (SEI) of Carnegie Mellon University and was developed by
the SEI, the Department of Defense and a host of other entities. For more information about
the SEI, please feel free to visit http://www.sei.cmu.edu/sei-home.html.
The CMM is designed to be an easy to understand methodology for ranking a company’s IT
related activities. The CMM has six levels 0 – 5 (see illustration below):
Level 0 – Not Performed
Level 1 – Performed Informally
Level 2 – Planned and Tracked
Level 3 – Well-Defined
Level 4 – Quantitatively Controlled
Level 5 – Continuously Improving
The purpose of these levels is to provide a “measuring stick” for companies looking to
improve their system development processes.
Figure 1: CMM Levels
Why is the CMM Valuable?
The CMM is valuable for several reasons. First, it is simple and easy to understand;
therefore, it speaks to the executives of a corporation. Many corporate executives are
already familiar with this model and it is probable that your company’s executives are also
familiar with it. Over the years I have been able to successfully use this model to illustrate
major IT issues/concepts to senior executives. I have found the CMM to be a very valuable
tool for attaining project funding for key initiatives like enterprise data asset management
and meta data repository development (I will talk about these concepts in greater detail as
my series on the CMM moves forward). Second, as you view the model it is intuitive that a
company cannot currently be ranked at a level 2 and directly jump to level 4. Instead, an
organization must first develop a strategy to elevate themselves to level 3. Third, many
large companies and government institutions are actively using this model to compare
themselves with other entities. In fact, many corporations have IT goals centered on the
CMM levels. Fourth, the model gives companies a mechanism to compare themselves with
other companies within their industry.
BRS Acceptance testing
SRS SystemmTesting
HLD System Integration testing
LLD Component Testing
Coding Unit Testing
Testing
CMM and CMMI
CMM = 'Capability Maturity Model', now called the CMMI ('Capability Maturity Model
Integration'), developed by the SEI. It's a model of 5 levels of process 'maturity' that determine
effectiveness in delivering quality software. It is geared to large organizations such as large U.S.
Defense Department contractors. However, many of the QA processes involved are appropriate
to any organization, and if reasonably applied can be helpful. Organizations can receive CMMI
ratings by undergoing assessments by qualified auditors.
Level 1 - characterized by chaos, periodic panics, and heroic efforts required by individuals to
successfully complete projects. Few if any processes in place; successes may not be
repeatable.
Level 2 - software project tracking, requirements management, realistic planning, and
configuration management processes are in place; successful practices can be repeated.
Level 3 - standard software development and maintenance processes are integrated throughout
an organization; a Software Engineering Process Group is is in place to oversee software
processes, and training programs are used to ensure understanding and compliance.
Level 4 - metrics are used to track productivity, processes, and products. Project performance is
predictable, and quality is consistently high.
Level 5 - the focus is on continouous process improvement. The impact of new processes and
technologies can be predicted and effectively implemented when required.
Perspective on CMM ratings: During 1997-2001, 1018 organizations were assessed. Of those,
27% were rated at Level 1, 39% at 2, 23% at 3, 6% at 4, and 5% at 5. (For ratings during the
period 1992-96, 62% were at Level 1, 23% at 2, 13% at 3, 2% at 4, and 0.4% at 5.) The median
size of organizations was 100 software engineering/maintenance personnel; 32% of
organizations were U.S. federal contractors or agencies. For those rated at Level 1, the most
problematical key process area was in Software Quality Assurance.
Sanity testing
Sanity testing is performed whenever cursory testing is sufficient to prove the application is
functioning according to specifications. This level of testing is a subset of regression testing.
It normally includes a set of core tests of basic GUI functionality to demonstrate connectivity to
the database, application servers, printers, etc.
Exhaustive Testing
Executing the program with all possible combinations of values for program variables. Feasible
only for small, simple programs.
