How to Select and Contract for Outsourced Agile

Development Services
Published 13 September 2023 - ID G00796216 - 32 min read
By Analyst(s): Luis Pinto, Gunjan Gupta, Jaideep Thyagarajan
Initiatives: IT Contracts Negotiations; Build a World-Class Software Engineering
Organization; IT Sourcing Strategy Development and Execution; Software Engineering
Practices

Organizations that need multidisciplinary development teams plan

to use more external custom software development providers.
This research shows sourcing, procurement and vendor
management leaders how to evaluate outsourced agile or DevOps
providers’ performance, quality and accountability.

Overview
Key Findings
■ When using outsourced software development providers for agile development,
organizations face significant governance challenges — for example, around delivery
responsibility, productivity commitment (trade-off between quality and velocity),
capacity management and skill availability.

■ Sourcing, procurement and vendor management (SPVM) leaders who only have
experience with fixed time and budget contracts may struggle to determine the
appropriate selection criteria for agile software development services.

■ Organizations that use traditional methods when managing incremental delivery

from a software development services provider continue to use legacy metrics and
fail to shift from a culture of control to one based on trust. This creates conflict in
relationships and provider governance.

Recommendations
SPVM leaders seeking to contract for custom software development services should:

Gartner, Inc. | G00796216 Page 1 of 21

This research note is restricted to the personal use of [email protected].

■ Review your organization’s agile maturity to identify the level of support and artifacts
required from an external service provider. Gain consensus with your IT delivery and
business stakeholders on an adaptive governance and flexible operating model such
as iterative delivery cycles, scope change and work location.

■ Select the right provider by using specific evaluation criteria based on your
stakeholders’ agile maturity and custom software development requirements. Use
evaluation criteria such as vendors’ expertise in agile methodologies, product-centric
approach, clear role definitions and internal and external teams’ responsibilities.
Assess vendors’ ability to execute using minimum viable products (MVP).

■ Meet your goals and objectives by focusing on outcomes when negotiating and
contracting for outsourced custom software development (CSD) services. Reduce
the risk of vendor underperformance by including agile and DevOps metrics in your
contract. Periodically review what has been delivered and the accuracy of the cost
and effort estimates.

Strategic Planning Assumption

By 2025, 95% of technology products will be developed using agile product
methodologies, up from 75% today.

Introduction
Fifty-four percent of those who responded to the 2022 Gartner IT Services Buyers survey
said that their organizations plan to source multidisciplinary teams to gain agile and
DevOps competencies. 1 Their top three objectives were to:

1. Better support new technology platforms.

2. Be in sync with emerging software delivery methodologies.

3. Increase productivity.

Top 5 Challenges to Changing the IT Services Outsourcing Delivery Model

While some organizations have begun to make meaningful progress to enhance their
baseline agile maturities, many still face significant barriers to advancing their business
agility. Here are the top five challenges organizations are facing when trying to change
their IT services outsourcing delivery model.

Gartner, Inc. | G00796216 Page 2 of 21

This research note is restricted to the personal use of [email protected].

 1. Procurement processes are rigorously prescriptive and are not adjustable.

2. Long-term agreements are currently in place with service providers.

3. Contractual life cycle management takes time to execute and is not in sync with the
dynamic and iterative nature of agile sprint cycles.

4. The financial chargeback process is unclear if agile programs span multiple

departments and projects.

5. Organizations have conflicting priorities between agile and waterfall delivery models
and do not usually have governance and communication processes in place.

In addition, some organizations do not have sufficient in-house skills and software
engineering capability, do not understand the changes required, and expect that an
outsourced partner can build the next best product, instead of a minimum viable product
(MVP).

Based on their internal capability, many clients rely totally on third-party providers to
perform tasks and governance and cannot control work delivery and outcomes in an agile
development environment. As a result, many clients fail to achieve successful provider
engagements, resulting in lost time and money.

This research provides sourcing, procurement and vendor management (SPVM) leaders
with a five-step process to contract for CSD services effectively. See Note 1 for Gartner’s
definition of the CSD services market.

Figure 1 shows the steps SPVM leaders should take to effectively contract for outsourced
agile software development services.

Gartner, Inc. | G00796216 Page 3 of 21

This research note is restricted to the personal use of [email protected].

Figure 1. How to Contract for Outsourced Agile Software Development Services

Analysis
Review Your Organization’s Agile Maturity
Organizations that are evolving in agile delivery should first evaluate the maturity of their
agile teams and work with service providers to define the roles and responsibilities of the
multidisciplinary team.

Figure 2 provides the agile maturity levels and the contracting models organizations
should choose based on their experience, team competencies and maturity of agile
adoption.

Gartner, Inc. | G00796216 Page 4 of 21

This research note is restricted to the personal use of [email protected].

Figure 2. Agile Maturity Levels and Contracting Models

Assess the agile maturity of your organization to understand how much support you will
need from an external agile provider. That will define the scope of services when creating
a provider proposal. Agile and DevOps operating models require significant commitment
from IT, business and the agile service provider and their roles and responsibilities must
be clearly defined to achieve business outcomes.

Here are the approaches you should take when scaling agile delivery, based on your agile
maturity level (as illustrated in Figure 1):

■ Early: Organizations with low agile maturity should apply agile offerings to a small
project, use on-site provider developers and get help from agile coaches to transition
to a new agile way of working. Teams that blend customer and supplier personnel
are most appropriate in this phase and should be trained on Scrum and Kanban
methodologies. The biggest challenge is to identify the right pilot project to get
started, so create a shortlist of projects that will cause little disruption to business
operations. The best contracting model for early adopters is time and materials.

Gartner, Inc. | G00796216 Page 5 of 21

This research note is restricted to the personal use of [email protected].

■ Growing: Organizations with medium agile maturity have some experience in
applying agile with limited outcomes. They can improve further by assessing team
structure, stakeholder contributions, process and technology adoption. To encourage
vendor accountability, everyone within a team except the product owner should
come from the same supplier. For more information on this, see Product Owner
Essentials. For long-term maintenance and to continue to grow internal capability,
we recommend using hybrid teams. At this level of maturity, you should consider a
managed-capacity contracting model where vendor teams are assigned specific
tasks and evaluated on unit-based metrics.

■ Confident: Organizations with significant agile maturity already have a DevOps

operating model in place — a combined support and delivery resource pool and a
demand-to delivery orchestration layer. As a result, they can take advantage of more
certainty in new contracts, making every sprint a fixed-price project. This approach
provides both sourcing and agilists with 85% of what they want, fully embraces agile
principles of flexibility and collaboration and provides sourcing with levers to
manage underperforming suppliers.

■ Mature: Organizations with a large custom engineering footprint should continue to

experiment with new contracting models (for example, outcome-based) focusing on
estimated business value and tying increased business value to features.

Select the Right Service Provider

Many organizations outsource custom development to external third-party service
providers and this has become an increasing trend. Almost all application outsourcing
service providers and systems integrators offer some level of product development
capability. However, some product management and DevOps or agile departments are tied
to some providers more deeply than others. The selection process should identify which
vendors bring actual in-depth experience in complex and large scale product development,
product management and agile.

Gartner, Inc. | G00796216 Page 6 of 21

This research note is restricted to the personal use of [email protected].

The selection process should also evaluate a provider’s maturity in agile coaching. This
should include adapting governance processes for agile and developing estimation
techniques based on the customer’s IT architecture, delivery model and complexity. These
governance processes should be driven by appropriate parameterized techniques, such as
complexity, size, and effort required for achieving or implementing a user story.
Organizations should consider applying different sourcing and procurement options. For
example, take advantage of competitive dialogues with a number of service providers to
deliver a co-created outcome-based managed service solution deal in a very short time
frame.

Use Agile Competitive Co-Creation and Design

Use dynamic sourcing to create a shortlist of three to five of the most suitable service
providers, then focus key objectives and critical success factors of both the solution and
the deal. The heart of this process is to deliver agile selection and negotiation, which
engages stakeholders to achieve their targets through a competitive dialogue with a few
credible providers. It uses competitive co-design to deliver critical outcomes through sprint
cycles spanning 90 days while negotiating in parallel.

The main benefits of this approach include:

■ Accelerating existing deal renewals.

■ Redesigning a failing deal.

■ Going to the market with a business problem versus using a traditional RFI or RFP.

For more information on this process, see Quick Answer: What Is Gartner’s Dynamic
Sourcing Process?

Key Elements to Selecting the Right Service Provider

IT sourcing managers can evaluate expertise, to some extent, by using simple metrics
such as the number of certified Scrum Masters and years of experience with agile
projects. Other organizations are being more creative — some have even run hackathons
as part of the supplier selection process.

SPVM leaders can build a detailed selection criteria using the evaluation criteria listed in
Table 1.

Gartner, Inc. | G00796216 Page 7 of 21

This research note is restricted to the personal use of [email protected].

Table 1: Key Elements to Select the Right Service Provider for Custom Software
Development Services
(Enlarged table in Appendix)

Gartner has published the Magic Quadrant for Custom Software Development Services,
Worldwide and the Critical Capabilities for Custom Software Development Services,
Worldwide, which offer details on global and pureplay service providers in this market.
The Magic Quadrant evaluates the worldwide capabilities of 20 service providers in the
CSD market and includes 10 honorable mentions. It provides details on execution and
vision while the Critical Capabilities evaluates these vendors on three use cases (unique
use, unique operational processes and unique products).

Mix and Match Vendors’ Products to Achieve the Optimum Collaboration Environment

Gartner, Inc. | G00796216 Page 8 of 21

This research note is restricted to the personal use of [email protected].

IT sourcing managers can encourage competition between service providers and across
multiple projects, but should contract with a single provider per product based on
business capabilities and customer value or experience. The goal should be to keep at
least two, if not three, service providers in play and award different deals to each.

If one service provider fails, others that are familiar with your organization will be ready to
take its place. However, using more than three service providers dilutes the spending with
each company and will impact the volume discounts they may offer. It could also affect
their attention and motivation to always engage with their best team. Finally, using too
many providers also increases the amount of time and resources needed to manage
multiple vendors and continuous improvement.

Negotiate and Adapt the CSD Services Contract

While most sourcing organizations have standardized contract templates, there are three
common difficulties in applying these to CSD projects:

■ Standardized templates are typically designed for legacy IT projects of all shapes,
sizes and scopes, which makes them very long. Negotiating these terms and
conditions is time-consuming and complex and can delay urgent business
initiatives.

■ These templates tend to be based on the assumption that the scope of work is
clearly defined and fixed at the time the contract is signed.

■ In CSD, contracts are more likely based on an initial or flexible scope rather than a
fixed and detailed one. The contract includes obligations and breach scenarios, but
the focus is more on product rather than on process, promoting a gain-or-pain
performance approach.

Focus on Outcomes and Customer and Provider Interactions

Clarify Quality Standards

A CSD services contract should be clear on quality standards and thresholds and state
when program backlog items are ready to enter sprint planning, through a definition of
ready. This describes requirements, stating when it is ready to be deployed through a
definition of done (DoD) that sets out what indicates that the product is production-ready
and what level of defects is acceptable postrelease.

Gartner, Inc. | G00796216 Page 9 of 21

This research note is restricted to the personal use of [email protected].

The customer needs the ability to hold a supplier accountable for quality. Accountability
should be proportional to the scope of the work and depend on the customer fulfilling its
own obligations, such as the design quality (if the customer is responsible).

Implement a New Contract Template

There should be a blanket statement of work (SOW) created for the entire program with a
not-to-exceed price, while a work order is created at a sprint level (see Figure 3).

Deliverables and expectations should be defined once you and the service provider have
agreed on the product backlog and DoD, ideally for each iteration or sprint, based on a
simple (one page) work order. For more information on this, see Quick Answer: What
Should a Definition of Done Contain? The service provider is paid based on the successful
completion of sprint deliverables at each work order level.

Figure 3. Software Development Services Contract Structure and Roles and

Responsibilities

Gartner, Inc. | G00796216 Page 10 of 21

This research note is restricted to the personal use of [email protected].

Each product must be managed through a single backlog. A single product backlog
maintains an ordered list of epics, features and stories needed to improve the combined
product delivery and value.

Create a RACI Matrix

Defining distinct roles and responsibilities helps avoid confusion among different project
members, so be sure to include a clear responsible, accountable, consulted and informed
(RACI) matrix. Table 2 describes who does what and when.

Table 2: RACI Matrix

(Enlarged table in Appendix)

Define Risk Principles Upfront and Manage Through Governance

Gartner, Inc. | G00796216 Page 11 of 21

This research note is restricted to the personal use of [email protected].

Agree with the supplier upfront on principles that explicitly describe how risk will be
shared. Define a governance mechanism that requires the parties to come together if the
speed at which working software is being delivered or its quality falls below what was
agreed on.

If the problem is the supplier’s fault, the customer’s recourse needs to be clearly defined,
such as an increase in supplier capacity at no cost, a financial credit or the right to
terminate the contract. However, if quality and performance are higher than the targets,
there should be an incentive to encourage the service provider to continue outperforming
contracted goals.

Use the following guidelines when selecting the contract model to fit your way of working
and delivery maturity:

■ Competitiveness: Service costs remain competitive throughout the contract duration.

■ Predictability: Service costs are predictable and clearly defined.

■ Granularity: Cost and service components are sufficiently detailed to allow flexibility.

■ Business alignment:

■ The SLA is aligned to real business needs.

■ Incentives and bonuses are used and measured against their business
impacts.

■ Productivity: Services are continuously improved throughout the contract term using
the latest transformative technologies.

Gartner, Inc. | G00796216 Page 12 of 21

This research note is restricted to the personal use of [email protected].

■ Shared gain or pain:

■ This approach replaces the typical incentive or penalty method widely used in
traditional contracts, to promote and foster collaboration.

■ A typical pain or gain model implementation uses a discounted fixed price per
unit of work plus discounted time and materials, starting with an agreed upon
an estimation of effort that all parties agree to. The contract anticipates a
discounted rate per unit of work, such as price per function point and
discounted rate cards (cost per day per job profile) to calculate deviations from
previous estimates.

■ Supplier incentives: Pricing is based on underlying incentives. For example,

clients pay a premium on top of the agreed rate card if they overachieve
performance and quality metrics that push the supplier to deliver services
according to the contracted SLA.

■ Client incentives: Pricing is based on underlying incentives that push the client
to use the services responsibly.

Table 3 lists various charging mechanisms for agile projects and illustrates that usability
varies widely. Assess each pricing model based on how it aligns with your business
objectives, risk and agile maturity level. Validate the most promising models with cross-
functional leaders to select the most suitable pricing model.

Gartner, Inc. | G00796216 Page 13 of 21

This research note is restricted to the personal use of [email protected].

Table 3: Pricing Models for Custom Software Development Services
(Enlarged table in Appendix)

Identify Performance Metrics to Include in Your Contract

Gartner advises that clients set productivity targets and penalties for individual agile
software developers. To develop an agile culture, it is extremely important that the whole
team focuses on the product vision and desired business outcomes. For more
information, see Toolkit: Essential Metrics for Application Services. The whole team,
including business, IT, SPVM leaders and service providers, should maintain and review
the metrics on the product backlog, scrum board, burn-down charts and so on.

Performance Management Metrics

Gartner, Inc. | G00796216 Page 14 of 21

This research note is restricted to the personal use of [email protected].

When outsourcing CSD services, include a set of metrics that measure penalty and
rewards, cost performance indicators (CPIs) and a set of metrics for reporting purposes
that do not measure against penalty or reward KPIs. Sample CPI metrics include velocity,
burn-down rate, plan-to-done ratio. Sample KPI metrics include automation percentage
and test-cases-passed percentage.

In addition to metrics (and when your organization is using a service provider to augment
staff’s technical skills), add staffing SLAs — for example, attrition, key resources retention,
bench strength, resumes submitted on time, new resources induction and certifications.
The benefits of a comprehensive metric suite include:

■ Managing CSD service providers more effectively.

■ Helping customers define delivery KPIs that are aligned to business goals and
objectives.

■ Using flow metrics to identify opportunities to improve delivery performance.

■ Helping to manage vendors with the right balance of diverse metrics.

Table 4 provides the key metrics (SLAs) clients should consider.

Gartner, Inc. | G00796216 Page 15 of 21

This research note is restricted to the personal use of [email protected].

Table 4: SLAs for Productivity and Quality in CSD Services, and Measurement Formulas
(Enlarged table in Appendix)

Continuous Improvement Measures

Implementing continuous improvement requires a structured approach that involves all
product team members and fosters a culture of experimentation, learning and innovation.
Here are four actions Gartner recommends you include in a continuous improvement
approach.

1. Identify what you want to achieve with continuous improvement and define clear,
measurable goals and objectives.

2. Create a baseline across the engineering group (internal and external teams) to
assess improvements.

Gartner, Inc. | G00796216 Page 16 of 21

This research note is restricted to the personal use of [email protected].

 3. Agree on improving individual metrics quarter-over-quarter in new contracts and
make service providers responsible for defining a plan to evolve.

4. Include in the contract feedback from your stakeholders about the product’s or
service’s quality and value, such as Net Promoter Score (NPS). Incorporate feedback
from team members on the effectiveness of the continuous improvement process.

Manage Underperformance
Agile delivery can also go wrong, sometimes due to supplier underperformance, which has
been one of the top challenges clients have faced recently. Therefore, what actions should
SPVM leaders take when a supplier is underperforming?

According to the principles behind the Agile Manifesto, the highest priority of agile
software development is “to satisfy the customer through early and continuous delivery of
valuable software.” 2 Despite this, much has been written about using metrics and targets
on agile projects (see Use the Right Metrics in the Right Way for Enterprise Agile Delivery).
SPVM professionals are often tempted to set hard targets for developer productivity and
software quality, incentivizing suppliers to meet these SLAs by imposing financial
penalties if they don’t.

This approach is all but impossible for product development, since there are no industry-
standard objective measures of developer productivity on which SLAs can be based. The
situation is not much better for quality measures such as defect density because they too
rely on a unit of measure of developer output. In addition, a major disadvantage of SLAs
in software development is that they distract the team (and the supplier) from providing
useful software to the end user.

As a result, organizations usually identify underperforming suppliers based on the

following characteristics:

■ Lack of or inconsistent communication (timely updates, progress reports and

feedback).

■ Inadequate flexibility to adapt quickly to business priorities and changes in

requirements.

■ Poor adherence to agile principles — for example, continuous delivery, iterative

development and customer collaboration and feedback.

There are three steps that SPVM leaders can take when a supplier is underperforming:

Gartner, Inc. | G00796216 Page 17 of 21

This research note is restricted to the personal use of [email protected].

■ Step No. 1: Raise the issue with the supplier and discuss it openly. As with all agile
projects, there should be a sprint retrospective that encourages the Scrum team to
suggest how the team can do better next time. This may require changes in both the
supplier’s and the customer’s behavior. The transition to agile is a major cultural
change. It is common for a newly formed team to take five or six sprints to
understand each other and learn to estimate their work capacity accurately.
Therefore, the first step should always be constructive dialogue with the supplier.
Major or consistent deviations from baseline KPIs should trigger a root cause
analysis and associated improvement plan.

■ Step No. 2: Take the vendor’s fees, which would have been put at risk by SLAs, and
use them more constructively. A percentage of the vendor’s fees can be held back
until the customer has achieved an important milestone. For example, consider an
agile team that operates two-week sprints but does not release to production until
they complete the fourth sprint. The customer could retain a percentage of the
vendor’s fees until deployment. The size of the retention and the length of the
retention period must be negotiated individually for each contract. A 5% retention for
one month will be much easier to negotiate than a 100% retention for two years.
Twenty percent for up to three months would be a reasonable compromise. This
aligns both the customer’s and vendor’s interest. Neither is happy until a MVP is
produced that meets mutually agreed on key success criteria for the new solution.

■ Step No. 3: If neither open discussion nor holding back fees works, you may need to
replace the supplier. Clearly, this is not an option that should be used often or
frivolously. Changing suppliers will delay the whole project while a new supplier is
found and brought up to speed. There will be additional cost while the old supplier
hands over to the new. However, this option needs to be kept in play throughout the
life of the contract, partly in case it is needed and partly as an incentive to the
supplier to perform well. It is extremely important that the master contract with the
supplier does not commit the customer to a fixed spend or a fixed duration. Even
though the customer has no intention of switching suppliers, the contract should
leave that possibility open as a form of leverage. This is in place of contractual
mechanisms such as ensuring the work each sprint completes in a way that is
maintainable and developable by another team. Examples of the contractual
mechanisms are: deliverables such as change documentation, SLAs and penalties.

Special thanks to Srini Pulugurtha for his contribution to this research.

Gartner, Inc. | G00796216 Page 18 of 21

This research note is restricted to the personal use of [email protected].

Evidence
1
2022 Gartner IT Services Buyers Survey. This study was conducted to understand how
the delivery and engagement model is evolving for IT services, how organizations are
introducing sustainability as a criterion for selecting IT services providers and the impact
on IT service budgets in the next 12 months. The research was conducted online from
August through October 2022 among 626 respondents from North America (n = 237),
Asia/Pacific (n = 169), Western Europe (n = 168) and Latin America (n = 52) with at least
$50 million in annual revenue.

Organizations were from most industries, except agriculture, real estate, services,
nonprofits, charities or NGOs and were outsourcing IT services (infrastructure,
applications or business process services) to a third-party vendor or contractor. An index
was created to classify organizations into traditional and progressive organizations based
on the stage of execution or plans to execute changes in the IT services outsourcing
delivery model.

Respondents were required to have been personally involved in service provider

identification, evaluation, selection or contracting in the last 24 months. Respondents
were required to have a job title of manager or above, depending on the functional areas
they were involved in (corporate or line of business leadership, application management,
enterprise architecture and technology innovation, finance, human resources,
infrastructure and operations, marketing, program and portfolio management and
sourcing, or procurement or vendor management).

Disclaimer: Results of this survey do not represent global findings or the market as a
whole, but reflect the sentiments of the respondents and companies surveyed.

2
Principles Behind the Agile Manifesto

Acronym Key and Glossary Terms

Multidisciplinary A multidisciplinary development team includes various
Development specialists such as security, testing, software developers and
Team user design to help develop each iteration of the software.

Note 1: Custom Software Development Services

Gartner defines the market for CSD services as development and iteration of custom
applications and software products specifically for an organization to satisfy its unique
business needs. These capabilities include:

Gartner, Inc. | G00796216 Page 19 of 21

This research note is restricted to the personal use of [email protected].

■ Gathering business requirements and coding the application from inception, building
it on a platform as a service (PaaS) or assembling it from existing web services or
other reusable pieces of code.

■ Integrating the developed application with other systems within the enterprise or with
external partner systems.

■ Incident resolution and support at Level 2 or Level 3 (L2/L3), correction of defects in

the software, and refactoring of technical debt for the developed software as it
moves into production.

■ The use of agile/DevOps in diversified, inclusive and multidisciplinary teams.

It excludes:

■ All activities related to implementing and maintaining enterprise commercial off-the-

shelf application suites such as SAP, Oracle, Workday and Salesforce.

■ Ongoing application management of legacy applications.

■ All business process outsourcing (BPO) related activities.

■ Any product revenue, such as the resale of software licenses and providers’ own or
third-party products.

■ Any physical (on-premises and cloud) compute assets associated with revenue.

■ Advisory consulting on business strategy and technical processes such as agile or

DevOps transformation.

■ Services that are marketed as “software product engineering” or “digital product

development” are likely to be good examples of the category Gartner defines as
“custom software development services.”

Document Revision History

5 Steps to Effectively Select and Contract With an Agile Development Service Provider - 3
October 2019

Recommended by the Authors

Some documents may not be available as part of your current Gartner subscription.

Gartner, Inc. | G00796216 Page 20 of 21

This research note is restricted to the personal use of [email protected].

Magic Quadrant for Custom Software Development Services, Worldwide
Critical Capabilities for Custom Software Development Services, Worldwide
How to Manage Blended Teams of In-House and Outsourced Software Engineering
Personnel
Drive High-Value LCAP Outcomes Through Strategic Partnerships With Service Providers
Quick Answer: How Can Software Engineering Leaders Communicate Business Cases
Effectively?
Quick Answer: How Can Software Engineering Leaders Reduce Time to Market?

© 2024 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of
Gartner, Inc. and its affiliates. This publication may not be reproduced or distributed in any form
without Gartner's prior written permission. It consists of the opinions of Gartner's research
organization, which should not be construed as statements of fact. While the information contained in
this publication has been obtained from sources believed to be reliable, Gartner disclaims all warranties
as to the accuracy, completeness or adequacy of such information. Although Gartner research may
address legal and financial issues, Gartner does not provide legal or investment advice and its research
should not be construed or used as such. Your access and use of this publication are governed by
Gartner's Usage Policy. Gartner prides itself on its reputation for independence and objectivity. Its
research is produced independently by its research organization without input or influence from any
third party. For further information, see "Guiding Principles on Independence and Objectivity." Gartner
research may not be used as input into or for the training or development of generative artificial
intelligence, machine learning, algorithms, software, or related technologies.

Gartner, Inc. | G00796216 Page 21 of 21

This research note is restricted to the personal use of [email protected].

Table 1: Key Elements to Select the Right Service Provider for Custom Software Development Services

Category Description and Capabilities Evaluation Criteria

Product Management Developing a product vision aligned to the Experience in:

organization’s business goals and objectives, ■ Understanding the customer journey or
including: hypothesis, user personas, market insights,
■ Setting up the product vision. product goals and features, KPIs and business
outcomes.
■ Developing the product strategic roadmap.
■ Controlling technical debt from the start and
■ Managing the product life cycle.
including technical debt resolution to improve
■ Using value stream metrics to improve agile maintainability and adaptability.
effectiveness.

Governance processes (such as security and Agile and product roles are clearly defined and
architecture reviews) align with agile ways of staff in new roles have the competencies, training
working. and support they need to be successful.
Agile teams effectively partner with IT and Is the service provider’s culture to have teams
corporate functions to achieve business colocated or distributed? If distributed, what are
outcomes. the collaboration tools and processes to build and
Funding is aligned to product teams and retain a team culture?
governance processes. Certifications and internal and external team
Funding governance is adapted appropriately. training that are relevant to the customer’s
Team organization and team members’ experience products and services.
in agile. Flexible funding governance processes in place
that enable reallocation of funds as priorities

Gartner, Inc. | G00796216 Page 1A of 15A

This research note is restricted to the personal use of [email protected].

 change.

Governance and Customer experience (CX) strategy and execution. User interface (UI) design and user experience (UX)
Roles and Responsibilities interaction functionality for custom software.
Experience in bringing in a human-centered
approach to experiences. This should be anchored
in understanding customers’ needs, rapid
prototyping, embedding service orchestration
capabilities for workflow automation and data and
analytics experience.

Technical architecture and cloud. Experience establishing a technical design

(blueprint), including software building blocks with
required functionality and interfaces.

AI or machine learning (ML) expertise. Experience in analyzing consumer buying behavior,

extracting data from scanned documents,
detecting patterns in pictures or videos or
translating speech into structured commands.
The ability to AI and ML models as part of a
custom software solution.

API integration and management. Skills in managing different integration scenarios:

real-time data, applications, batch data, event
streaming and B2B.
Expertise in the major enterprise integration
platforms as a service (iPaaS) and API
management.

Technical Expertise Analytics and business intelligence (BI) expertise. Experience in using BI applications and packaged

Gartner, Inc. | G00796216 Page 2A of 15A

This research note is restricted to the personal use of [email protected].

 solutions.

Quality engineering practices to help build custom The ability to provide static code analysis,
software with high levels of quality, reliability and automated testing, functional and nonfunctional
maintainability. testing and performance testing.

Software Engineering Approaches Develop software in rapid increments using agile, Teams embrace agile ways of working — fast,
DevOps and product centricity. flexible and collaborative — to achieve business
and customer outcomes.
Experience beyond Scrum — for example, Kanban,
large-scale Scrum (LeSS), Nexus, SAFe,
Scrum@Scale and so on.
Experience in key tech tools that facilitate agile and
DevOps ways of working, such as value stream
management platforms and DevOps platforms
(Atlassian, Microsoft Azure DevOps, open source
continuous integration and continuous delivery [CI]
and continuous deployment [CD] tool chains).

Pricing Pricing models used in CSD services contracts and Availability of alternative pricing models. For
key metrics. example:
■ Time and material (T&M).

■ Managed capacity.

■ Unit-based (user, application, sprint and MVP).

■ Fixed price per iteration (sprint and program
increment).

■ Business outcome-based.

Gartner, Inc. | G00796216 Page 3A of 15A

This research note is restricted to the personal use of [email protected].

Intellectual Property (IP) IP and assets developed to deliver CSD services. How does the service provider provide source code
and ownership and guarantee that the IP is not
from other sources such as GenAI or open source?
IP safeguards: assets to mitigate risks of misuse,
theft or reverse engineering.

Innovation Rapidly adopt and capitalize on innovative custom Innovative use of technology such as APIs, data
solutions developed by the service provider. and analytics, AI, cloud and multiexperience
development.
Collaboration with ecosystem partners to co-
innovate.

Source: Gartner (September 2023)

Gartner, Inc. | G00796216 Page 4A of 15A

This research note is restricted to the personal use of [email protected].

Table 2: RACI Matrix

Agile Delivery Team Product Owner Scrum Master Dev team Release Manager
(TA/UX/DEV/QA)

Prioritize the product A, R I C I

backlog (forms and
informs 3-month release
plan).

Select candidates for A, R I I I

the next sprint.

Sprint Planning Agree on the sprint A, C R R I

backlog and goal for the
next sprint.

Communicate the sprint A, R C C I

backlog goal and sprint
stakeholders’
expectations.

Inspect the work C A R N/A

completed since the last
daily scrum.

Daily Scrum Standup Plan the work to be C A R N/A

completed in the next 24
hours.

Gartner, Inc. | G00796216 Page 5A of 15A

This research note is restricted to the personal use of [email protected].

 Identify and resolve I A, R R N/A
development obstacles.

Conduct an in-sprint I A R I
review of progress (UX
testing, designs or
prototype).

Demo sessions Agree on the next steps A R R I

(UX testing, designs or
prototype and user
feedback).

Manage (prioritize) the A, R C, I I N/A

product backlog.

Refinement (Product Refine the product A R C N/A

Backlog) backlog based on the
initiative or user story
level.

Refine the ideation A, C C R N/A

workshop.

Refine the details and A R C N/A

estimates, making them
ready for play.

Review the commitment C A R I

progress in sprint

Gartner, Inc. | G00796216 Page 6A of 15A

This research note is restricted to the personal use of [email protected].

Sprint Review Revise the product A, R I C I
backlog (optional).

Communicate the A, R C C I
output of the sprint and
any impact on the
prioritized backlog.

Gather the Scrum Team I A, R C I

retrospective.

Sprint Retrospective Identify improvements I A, R C I

that can be
implemented.

Create a plan for I A, R C I

improvements and
communicate as
appropriate.

Footnote: (A)ccountable, (R)esponsible, (C)onsulted, (I)nformed

Source: Gartner (September 2023)

Gartner, Inc. | G00796216 Page 7A of 15A

This research note is restricted to the personal use of [email protected].

Table 3: Pricing Models for Custom Software Development Services

Pricing Model Description Attention Points

Time and Material Time and material (T&M) pricing is simple, plain It is difficult to predict cost fluctuations over time.
and widely adopted, mainly when using a service The risk is totally on the customer side.
provider to augment staff. The provider will use every opportunity to extend
Clients often have concerns about paying before the project or program for increased revenue.
seeing results, due to the iterative approach.
There are a number of T&M pricing variations to
reduce customer exposure and balance pain and
gain:
■ Capped T&M per project.

■ Capped T&M per iteration.

■ Capped T&M per iteration, plus adjustments.

Unit-Based Most agile teams measure progress in story points, It is very difficult to benchmark a price per story
which are difficult to measure, are different for point to verify value for money.
each team or provider and have no industry
standard.
Organizations should define story point estimation
techniques based on complexity and initial or
/minimum productivity based on its environment
and systems. They should strive to improve
productivity year over year to drive down costs.

Gartner, Inc. | G00796216 Page 8A of 15A

This research note is restricted to the personal use of [email protected].

Managed Capacity The supplier provides an entire team price or a As with T&M, the client bears the risk of the project,
fixed monthly price per team. This provides a blend while the service provider bears none.
of resources as managed capacity. To avoid this risk, client organizations should
There should be a set of quality and performance define a beginning level of quality and productivity
metrics associated with this model to ensure based on its environment and systems and strive
continued commitment from service providers. to improve productivity to drive down cost.
Payment can be held back if a service provider fails
to meet productivity metrics.

Fixed Price Per Iteration (Sprint, Program This is an effective hybrid approach that combines The provider adds the contingency.
Increment) the flexibility of agile development while expecting Price is based on the delivery team’s experience
the provider to demonstrate accountability. At the and skills.
start of each Sprint, the provider and client agree Part of the provider’s revenue will be at risk.
upon a set of stories or features, prioritized by Change requests for unambiguous scope
task. This gives vendor management an audit trail documentation can affect the price.
of provider commitments. Payment can be held
back if the provider fails to meet milestones, such
as deploying a minimum viable product.
If the scope is agreed on beforehand for each
iteration, the provider would add some
contingency to handle project and work variability.
However, given the iterative approach and limited
scope of each iteration, contingency tends to be
smaller and smaller.
If the scope is highly flexible, transparency,
frequent delivery and early termination can help.

Business Outcome-Based Although not widely adopted, many sourcing It is difficult to implement because defining an
organizations are interested in business outcome- outcome is not simple, and a business outcome

Gartner, Inc. | G00796216 Page 9A of 15A

This research note is restricted to the personal use of [email protected].

 based contracts, as we cover in Outcome-Driven may depend only partly on the service provider.
Metrics for the Digital Era. And since many
business outcomes may not be achieved until
many months after the project is completed, there
could be a delay between the service provider’s
costs and its revenue.
This pricing model requires a clear boundary for
executing the service provider scope. It should also
avoid dependencies as much as possible on the
customer and third parties for successfully
executing that scope.

Source: Gartner (September 2023)

Gartner, Inc. | G00796216 Page 10A of 15A

This research note is restricted to the personal use of [email protected].

Table 4: SLAs for Productivity and Quality in CSD Services, and Measurement Formulas

Service-Level Category Type Description Measurement Time or Service-Level Metric

Formula

Technical Debt Quality Measures the amount of Technical debt to fix the Between 3% and 5%
(Refactoring or Paying Down technical debt in the system or application (value
Debt in Small Increments) codebase and the team’s plan to be measured using clean
for managing the debt. core calculation). In other
words, measured through
reusability, portability,
security, and so on.

Burn-down Rate or Velocity Velocity or flow Tracks progress in completing Story points delivered per No common target.
the project backlog. sprint. Gradually increase velocity
Note: Story points are over time.
subjective and need to have a
common definition.

Planned Versus Actual Velocity or flow Measures total story points (A/P) x 100 Aim for 100%, but do not
completed versus those A = Story points accepted as consider less than that to be
planned. done during the sprint. a SLA breach.
P = Story points planned to be During the first three sprints,
completed during the sprint. failure to achieve 100%
should not be considered
underperformance.
After sprint 3, an occasional
failure to meet 100% should

Gartner, Inc. | G00796216 Page 11A of 15A

This research note is restricted to the personal use of [email protected].

 be discussed with the service
provider and remediated.
A continued and repeated
failure to meet 100% indicates
a serious problem with the
contract and should trigger a
discussion about early
termination.

Customer Lead Time Velocity or flow The total time the client is The duration of the delivered The goal should be to create
waiting for an item to be date to the requested date. short lead times for feature
delivered. The delivered date is the date delivery.
This measures the time from on which a feature is Lead time complements
when the customer request is deployed to production. velocity because it measures
added to the product backlog The requested date is the the entire agile system from
to when all work on the item date on which the business end to end.
is completed and the request requests a feature.
is delivered.

Cycle Time Velocity or flow Cycle time is the amount of The duration of the delivered The goal should be to create
time that the team spent date to the start date. short cycle times for feature
actually working on this item The delivered date is the date delivery, ideally deploying
(without the time that the on which a feature is functionality continuously to
task spent in the product deployed to production. customers.
backlog). The start date is the date on
which the team begins
working on a feature.

Overall Defect Rate Quality Indicates the effectiveness of (DDPR + DDT + DR)/ATD Defects per sprint: 50 defects

Gartner, Inc. | G00796216 Page 12A of 15A

This research note is restricted to the personal use of [email protected].

 the domain/technical DDPR = the number of per 1,000 hours (baseline)
expertise, review, and testing defects detected at peer or
activities performed during review. Defects per month, per
the course of the release. DDT = the number of defects person in production
detected during testing.
DR = the number of defects
reported in the release.
ATD = the actuals to date
(total days spent on the
product by that specific
team).

Cost of Quality Quality Indicates the percentage of (AEPA + ARE + ATE)/ATD x Status report (sprint end)
time spent on quality 100
activities against the total AEPA = the actual effort on
time spent on the release. process audits.
ARE = the actual rework
effort.
ATE = the actual training
effort.
ATD = the actuals to date is
the total effort spent in all
tasks till date during sprints.

Project Estimation (Actual Business value Indicates the percentage of Monthly 95% of all projects or sprints
Cost Versus Estimated Cost) the project or sprints above or complete at or under budget
below budget. and no projects or sprints
completed at 110% or more
over budget.

Gartner, Inc. | G00796216 Page 13A of 15A

This research note is restricted to the personal use of [email protected].

Business Outcomes and Business value Indicates if the deliverable Quarterly The goals should be:
Customer Satisfaction produces the agreed-upon ■ Achieve a positive trend in
business outcome from the customer satisfaction and
product roadmap. adoption.
Are the customers who are
■ Meet business business
using the product satisfied,
outcomes outlined in the
loyal and advocating for it?
product roadmap.

■ Reduce overall cost to

serve and cost per unit by
X%.

■ Increase growth or top line

revenue, number of
customers, number of
orders, and so on.

■ Reduce operating
expenses (opex) due to
adopting DevSecOps
practices and
automations.

Sustainability Business value Indicates if the product Quarterly Environment (reducing

development process meets carbon footprint and so on).
the company’s targets Social (improving health and
toward sustainable welfare, for example).
development.

Gartner, Inc. | G00796216 Page 14A of 15A

This research note is restricted to the personal use of [email protected].

 Governance (100% GDPR and
sovereign data compliance).
Adherence to security
standards (reduced data
exposure, easier user audit).
High-risk and medium-risk
means no violations.

Talent and People Organizational effectiveness Retaining resources. Quarterly Service provider’s team
Management Onboarding new resources. turnover is less than 20%.
Lead time for onboarding (on-
site, off-site and offshore) is
greater than 80%.

Footnote: Ensure that service providers use your pipeline and tooling. This will give you full access to all metrics automatically, enabling you to monitor
delivery and status.

Source: Gartner (September 2023)

Gartner, Inc. | G00796216 Page 15A of 15A

This research note is restricted to the personal use of [email protected].