SUSTAINABLE

­DEVELOPMENT
METHODOLOGY
Assessing the environmental,
social and economic impacts
of policies and actions

ICAT SERIES OF
ASSESSMENT GUIDES
 How to use the Assessment Guides

This guide is part of a series developed by the Initiative for Climate

Action Transparency (ICAT) to help countries assess the impacts of
© April 2020
policies and actions. It is intended to be used in combination with
other ICAT assessment guides and can be used in conjunction with
All rights reserved. No part of this
other guidance.
publication may be reproduced, stored
in a retrieval system or transmitted, in
any form or by any means, electronic,
photocopying, recording or otherwise, SERIES OF ICAT ASSESSMENT GUIDES
for commercial purposes without prior
Introduction to the ICAT Assessment Guides
permission of UNOPS. Otherwise, material
in this publication may be used, shared,
copied, reproduced, printed and/ or stored, Impact assessment guides
provided that appropriate acknowledgment
is given of UNOPS as the source and Greenhouse gas
impacts:
copyright holder. In all cases the material
may not be altered or otherwise modified Renewable Energy Sustainable
without the express permission of UNOPS. Development

Recommended citation: ICAT (Initiative Transport Pricing

for Climate Action Transparency) (2020).
Sustainable Development Methodology: Transformational
Forestry
Assessing the Environmental, Social and Change
Economic Impacts of Policies and Actions,
D. Rich, R. Song and K.H. Olsen eds. Agriculture
Washington D.C.: World Resources Institute;
Copenhagen: UNEP DTU Partnership. Non-State and
Buildings Efficiency Subnational
https://climateactiontransparency.org/ Action
icat-toolbox/sustainable-development

ICAT Donors Process guides

Federal Ministry for the

Environment, Nature Conservation Stakeholder Technical
and Nuclear Safety Participation Review
Contents

Part I: Introduction, objectives and key concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

2 Objectives of assessing sustainable development impacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

3 Key concepts, steps and planning the assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Part II: Defining the assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4 Defining the policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

5 Choosing which impact categories and indicators to assess . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

Part III: Qualitative approach to impact assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

6 Identifying specific impacts within each impact category . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

7 Qualitatively assessing impacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

Part IV: Quantitative approach to impact assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

8 Estimating the baseline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

9 Estimating impacts ex-ante . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

10 Estimating impacts ex-post . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

11 Assessing uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

Part V: Monitoring and reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

12 Monitoring performance over time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137

13 Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

Part VI: Decision-making and using results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

14 Evaluating synergies and trade-offs, and using results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

Appendix A: Example of quantifying the impact of a solar PV incentive policy . . . . . . . . . . . . . . 177

Appendix B: Stakeholder participation during the assessment process . . . . . . . . . . . . . . . . . . . . . 195
Appendix C: Qualitative research methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197

Abbreviations and acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

Glossary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 206
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211
Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 214
PART I
Introduction, objectives and key concepts
1 Introduction

Governments around the world are increasingly The methodology was developed with the following
focused on implementing policies and actions that objectives in mind:
achieve sustainable development and climate change
objectives in an integrated manner. In this context, there • to help users assess all relevant sustainable
is an increasing need to assess and communicate the development impacts of policies and actions
multiple impacts of policies and actions to ensure that in an integrated way
they are effective in delivering a variety of sustainable
development and climate change benefits. Policy • to help policymakers and other decision
assessment can help countries more effectively achieve makers develop effective strategies for
the objectives of both the Paris Agreement and the achieving sustainable development objectives
2030 Agenda for Sustainable Development, including through a better understanding of the various
the Sustainable Development Goals (SDGs). There impacts of policies and actions
is an urgent need to transition towards sustainable
development and net zero global greenhouse gas (GHG) • to support consistent and transparent
emissions, as underlined in the special report Global reporting of sustainable development impacts
Warming of 1.5°C1 by the Intergovernmental Panel on and policy effectiveness.
Climate Change (IPCC).
This methodology supports multiple objectives
users may have (elaborated in Chapter 2), including
advancing policies and actions that contribute to
1.1 Purpose of the methodology multiple SDGs and priorities, building support for
climate actions by assessing and communicating the
The purpose of this methodology is to help users impacts that are most relevant to national audiences,
assess the sustainable development impacts of and informing policy design and implementation to
policies and actions. Sustainable development maximize positive impacts across multiple impact
impacts include a wide variety of impacts across categories.
three dimensions: environmental impacts, social
impacts and economic impacts. Examples of impacts The methodology is intended to help policymakers
include improved health from reduced air pollution, and analysts systematically assess multiple
job creation, poverty reduction, increased energy sustainable development and climate change
access, gender equality, and many others (further impacts to help achieve the objectives of both the
elaborated in Chapter 5). SDGs and the Paris Agreement. Assessing a broad set
of impacts before and after policy implementation
This methodology helps users answer the following can help policies be more effective and durable,
questions: generate positive benefits for society, and achieve
desired climate and development outcomes. This
• What sustainable development impacts is type of assessment can help integrate SDGs and
a given policy or action likely to have in the climate targets into a unified process – for example,
future? by identifying and reporting on the sustainable
development benefits of actions taken to achieve
• Is a given policy or action on track and nationally determined contributions (NDCs) under
delivering expected results? the Paris Agreement. It may also facilitate increased
access to climate finance, given the inclusion of
• What impacts has a given policy or action had sustainable development priorities in the United
to date? Nations Framework Convention on Climate Change
(UNFCCC), the Paris Agreement and the Green
Climate Fund.

1
Available at: www.ipcc.ch/sr15/.
4 Sustainable Development Methodology

1.2 Relationship to other 1.3 Intended users

methodologies and resources
This methodology is intended for use by a wide range
This methodology is part of the Initiative for Climate of organizations and institutions. Throughout this
Action Transparency (ICAT) series of guides for document, the term “user” refers to the entity using
assessing the impacts of policies and actions.2 the methodology.
It is intended to be used in combination with
other ICAT guides that users choose to apply. The The following examples explain how different types
series of assessment guides is intended to enable of users can use the methodology:
users who choose to assess GHG, sustainable
development and transformational impacts of a • Governments. Assess the environmental,
policy to do so in an integrated and consistent way social and economic impacts of policies and
within a single impact assessment process. Users actions to inform and enhance policy design
of this methodology should also consult the ICAT and implementation, improve monitoring of
Stakeholder Participation Guide3 on how to carry out progress of implemented policies and actions,
effective stakeholder participation when designing, retrospectively evaluate impacts to learn from
implementing and assessing policies and actions, experience, report on progress towards SDGs,
including when assessing sustainable development and facilitate access to financing for policies
impacts using this methodology. Refer to the ICAT and actions.
Introductory Guide for more information about the
ICAT assessment guides and how to apply them in • Donor agencies and financial institutions.
combination. Assess the impacts of finance provided, such
as grants or loans, to support sustainable
This methodology is informed by existing resources development policies and actions, including
such as the Greenhouse Gas Protocol Policy and results-based financing and development
Action Standard (© WRI 2014; all rights reserved)4 and policy loans.
the Framework for Measuring Sustainable Development
in NAMAs (UNEP DTU Partnership and IISD 2015).5 • Businesses. Assess the impacts of
The methodology draws on the Policy and Action private sector actions, such as voluntary
Standard, which provides guidance on estimating the commitments, implementation of new
GHG impacts of policies and actions, by following technologies and private sector financing, or
the same basic structure and series of steps and assess the impacts of government policies and
using many of the same concepts, where they are actions on businesses and the economy.
relevant to assessing sustainable development
impacts. Figures and tables adapted or reproduced • Research institutions and non-
from the Policy and Action Standard are cited, but for governmental organizations (NGOs). Assess
readability not all text taken directly or adapted from the environmental, social and economic
the Policy and Action Standard is cited. In addition impacts of policies and actions to evaluate
to the basic structure and steps, specific elements performance or provide support to decision
drawn from the Policy and Action Standard include makers.
the assessment principles and key concepts 3.1.3-
4 and 3.1.7-8 (Chapter 3), describing the policy or • Stakeholders affected by policies and
action (Chapter 4), the approach to identifying policy actions, such as local communities and
impacts and determining significance (Chapters 6 civil society organizations. Participate more
and 7), the framework for quantifying impacts effectively in the design, implementation
(Chapters 8–11), and the glossary. This methodology and assessment of policies and actions to
is consistent with the Policy and Action Standard and ensure that their concerns and interests are
can be used in parallel with it. addressed.

2
Available at: https://climateactiontransparency.org/icat-toolbox.
3
Available at: https://climateactiontransparency.org/icat-toolbox.
4
Available at: www.ghgprotocol.org/policy-and-action-standard.
5
Available at: https://unepdtu.org/publications/framework-for-
measuring-sustainable-development-in-namas.
 Part I: Introduction, objectives and key concepts 5

1.4 Scope and applicability • in any sector, such as agriculture, forestry,

of the methodology energy, transport, industry and waste, as well
as cross-sector policy instruments
This methodology provides an overarching
framework and process for assessing sustainable • that are planned, adopted or implemented
development impacts of policies.6 It provides
general principles, concepts and procedures that • that are new policies; or extensions,
are applicable to all types of policies and actions, all modifications or eliminations of existing
sectors, and all types of sustainable development policies.
impacts. It does not provide specific guidance for
individual impact categories, such as jobs, air quality As the methodology is developed under ICAT, its
or health, or prescribe specific calculation methods, focus is on assessing the sustainable development
tools or data sources. Other guidelines, methods and impacts of policies that have an impact on climate
tools can be used in combination that provide more change. These include policies implemented
in-depth methods for specific impact categories, such primarily to achieve climate goals, as well as
as air quality and health, or that focus specifically on policies primarily implemented to achieve other
economic, social or environmental impacts (see the environmental, social or economic objectives, but
ICAT website7 for a list of complementary resources). that have an impact, either positive or negative, on
GHG emissions.
This document is organized into six parts (Figure 1.1).
Part I provides an introduction, including objectives, Table 1.1 presents general types of policies that may
key concepts and steps. Part II provides guidance be assessed. The list is not exhaustive, and some
on defining the assessment. Part III provides users may have policies of other types.
a qualitative approach to impact assessment,
and Part IV provides a quantitative approach to
impact assessment. Parts III and IV cover both ex-
ante (forward-looking) assessments and ex-post
(backward-looking) assessments. Part V covers
monitoring and reporting, and Part VI provides
guidance on decision-making and using results.

1.4.1 Types of policies and actions

In this methodology, “policy or action” refers to

interventions taken or mandated by a government,
institution or other entity. These can include laws,
directives and decrees; regulations and standards;
taxes, charges, subsidies and incentives; information
instruments; voluntary agreements; implementation
of technologies, processes or practices; and public or
private sector financing and investment.8

The methodology is applicable to policies:

• at any level of government (national,

subnational, municipal) in all countries and
regions

6
Throughout this document, where the word “policy” is used
without “action”, it is used as shorthand to refer to policies and
actions, and policies and measures. See Glossary for definition of
“policy or action”.
7
https://climateactiontransparency.org/icat-toolbox/sustainable-
development.
8
WRI (2014).
6 Sustainable Development Methodology

FIGURE 1.1

Overview of the methodology

Part I: Introduction, objectives and key concepts

Understand the purpose and applicability of the methodology (Chapter 1)

Determine the objectives of the assessment (Chapter 2)
Understand key concepts and steps, and plan the assessment (Chapter 3)

Part II: Defining the assessment

Clearly define the policy to be assessed (Chapter 4)

Choose which impact categories and indicators to assess (Chapter 5)

Part III: Qualitative approach to impact assessment

Identify specific impacts of the policy within chosen impact categories (Chapter 6)
Qualitatively assess each specific impact (Chapter 7)

Part IV: Quantitative approach to impact assessment

Estimate baseline values for impacts included in the quantitative assessment boundary (Chapter 8)
Estimate policy scenario values for the same impacts (ex-ante) (Chapter 9)
Estimate policy scenario values for the same impacts (ex-post) (Chapter 10)
Assess uncertainty (Chapter 11)

Part V: Monitoring and reporting

Monitor the performance of indicators over time (Chapter 12)

Report the results and methodology used (Chapter 13)

Part VI: Decision-making and using results

Evaluate synergies and trade-offs, and decice which policies to implement (Chapter 14)
 Part I: Introduction, objectives and key concepts 7

TABLE 1.1

Types of policies

Type of policy Description

Regulations and Regulations or standards that specify abatement technologies (technology regulation or
standards standard), or minimum requirements for energy consumption, pollution output or other activities
(performance regulation or standard). They typically include penalties for non-compliance.

Taxes and charges Levies imposed on each unit of activity by a source – for example, a fuel tax, carbon tax, traffic
congestion charge, or import or export tax.

Subsidies and Direct payments, tax reductions, price supports or the equivalent provided by governments to an
incentives entity for implementing a practice or performing a specified action.

Voluntary Agreements, commitments or actions undertaken voluntarily by public or private sector

agreements or actors, either unilaterally or jointly in a negotiated agreement. Some voluntary agreements
actions include rewards or penalties associated with participating in the agreement or achieving the
commitments.

Information Requirements for public disclosure of information. They include labelling programmes, reporting
instruments programmes, rating and certification systems, benchmarking, and information or education
campaigns aimed at changing behaviour by increasing awareness.

Emissions trading Programmes that establish a limit on aggregate emissions of various pollutants from specified
programmes sources; require sources to hold permits, allowances or other units equal to their actual
emissions; and allow permits to be traded among sources. These programmes are also referred
to as emissions trading systems or cap-and-trade programmes.

Research, Policies aimed at supporting technological advances, through direct government funding or
development and investment, or facilitation of investment, in technology research, development, demonstration and
deployment policies deployment activities.

Public procurement Policies requiring that specific attributes (such as social or environmental benefits) are considered
policies as part of public procurement processes.

Infrastructure Provision of (or granting a government permit for) infrastructure, such as roads, water, urban
programmes services and high-speed rail.

Implementation Implementation of technologies, processes or practices (e.g. those that reduce emissions
of technologies, compared with existing technologies, processes or practices).
processes or
practices

Financing and Public or private sector grants or loans – for example, those supporting development strategies
investment or policies (e.g. development policy loans or development policy operations such as loans, credits
and grants).

Source: Adapted from WRI (2014), based on IPCC (2007).

8 Sustainable Development Methodology

Policies may refer to interventions at various levels 1.4.2 Flexible approach

of detail, from broad strategies, plans or goals that
define high-level objectives or desired outcomes; This methodology provides flexibility in how to
to specific policy instruments to carry out a broad assess the sustainable development impacts of
strategy, plan or goal; to the implementation of policies, to enable users to apply it in the context
technologies, processes or practices (sometimes of their own objectives and available resources.
called “measures”) that result from policy It provides guidance rather than requirements
instruments. These are illustrated in Figure 1.2, and is non-prescriptive, to accommodate various
which shows the range of interventions, from more national circumstances. Users do not need to follow
aspirational to more concrete. all steps, but instead can follow just the steps that
are relevant to their own needs. Each step can
This methodology is primarily designed to assess be implemented using a more simplified or more
specific policy instruments, and the implementation sophisticated approach, depending on availability of
of technologies, processes and practices. Users who data and resources, and user objectives. Different
intend to assess the effects of broad strategies, options for applying the methodology, including
plans or goals should first define the individual policy whether to follow a qualitative or a quantitative
instruments – or technologies, processes or practices approach, are explained in Chapter 3. Certain
– that will be implemented to achieve the strategy objectives may call for greater accuracy, consistency
or plan. Broad strategies or plans can be difficult and transparency in the way impacts are assessed
to assess because the level of detail needed to and reported, such as accessing financing or
assess impacts may not be available without further reporting on progress towards the SDGs and the
specificity, and different policies used to achieve the Paris Agreement.
same goal could have different impacts.
As a result of this flexibility, users applying the
The methodology is primarily designed for actions methodology and readers of the resulting impact
at a larger scale than individual projects. The focus assessment reports should be aware of potential
is on policies and actions, given the ongoing shift uncertainties when interpreting the results. Users
to broader policies and actions as represented who intend to compare or aggregate the results
by countries’ NDCs. However, users assessing the of multiple impact assessments should be aware
impacts of individual projects may also find the that differences in reported results may be a
methodology helpful. result of different methodological choices, rather

FIGURE 1.2

Types of interventions

TYPE OF INTERVENTIONS EXAMPLE APPLICABILITY OF

­METHODOLOGY

Partial; should further define

Broad strategies, Intent to increase energy
the specific policy instruments to
plans or goals efficiency by 30% by 2030
achieve the broad strategy

Energy efficiency standard

Policy instruments Applicable
for appliances

Implementation of
Replacement of old appliances
technologies, processes Applicable
with new, efficient ones
or practices
 Part I: Introduction, objectives and key concepts 9

than real-world differences. For example, two performance of key performance indicators,
assessments of the impacts of a policy on jobs and and expected future impacts of a policy
economic development may come to two different
conclusions as a result of differences in methods and • after implementation – to assess what
assumptions. To help overcome this challenge, this impacts have occurred as a result of a policy
methodology encourages transparent reporting (in (ex-post assessment).
Chapter 13) to explain the methods and assumptions
used, to help ensure that results are properly Depending on the objectives and when the
interpreted. methodology is applied, users can follow the steps
for ex-ante assessment, ex-post assessment or
both. The most comprehensive approach is to
apply the methodology before implementation,
1.5 When to use the methodology regularly during policy implementation and again
after implementation. Users carrying out an ex-post
The methodology may be used at multiple points assessment only can skip Chapter 9. Users carrying
throughout the policy design and implementation out an ex-ante assessment only can skip Chapter 10.
process, including:
Figure 1.3 outlines a sequence of steps to monitor
• before implementation – to assess the and assess impacts at multiple stages in a policy
expected future impacts of a policy (ex-ante design and implementation cycle. In the figure, the
assessment) process is iterative, such that insights from previous
experience inform improvements to policy design and
• during implementation – to assess implementation, and the development of new policies.
the impacts achieved to date, ongoing

FIGURE 1.3

Assessing impacts during a policy design and implementation cycle

Define
policy
objectives

Identify
Assess potential policies
impacts and assess their
ex-post impacts
ex-ante

Monitor
Select and
progress
implement
during policy
policies
implementation

Source: Adapted from WRI (2014).

10 Sustainable Development Methodology

1.6 Key recommendations To help users apply the methodology, the ICAT
website10 provides a list of calculation tools, models
The methodology includes key recommendations and resources for estimating the social, economic
that are recommended steps to follow when and environmental impacts of policies, organized
assessing and reporting impacts. These by impact category. These supplemental resources
recommendations are intended to help users provide more detailed methods for various impact
to produce credible and high-quality impact categories.
assessments that are based on the principles of
relevance, completeness, consistency, transparency
and accuracy.
1.9 Process for developing
Key recommendations are indicated in subsequent the methodology
chapters by the phrase “It is a key recommendation to
…”. All key recommendations are also compiled in a This methodology has been developed through
checklist at the beginning of each chapter. an inclusive, multi-stakeholder process convened
by ICAT. The Sustainable Development Methodology
Users who want to follow a more flexible approach is led by the World Resources Institute (lead) and
may choose to use the methodology without UNEP DTU Partnership (co-lead), who serve as the
adhering to the key recommendations. The ICAT secretariat and guide the development process.
Introductory Guide provides more information on The first draft was developed by drafting teams,
how and why key recommendations are used consisting of a subset of a broader Technical
within the ICAT methodology documents, and Working Group (TWG) and the secretariat. The TWG
on following either the “flexible approach” or the consists of experts and stakeholders11 from a range
“key recommendations approach” when using the of countries identified through a public call for
methodology. Refer to the Introductory Guide before expressions of interest. The TWG contributed to the
deciding which approach to follow. development of the first draft through participation
in regular meetings and written comments. A Review
Group provided written feedback on the first draft.

1.7 Alignment with Sustainable The second draft was applied by ICAT participating
Development Goals countries and other non-state actors to ensure that
it could be practically implemented. This version
This methodology is informed by, and compatible of the methodology was informed by the feedback
with, the United Nations SDGs9 and is intended to gathered from that experience and includes case
help users assess the impact of policies in relation studies from those applications.
to the SDGs. Chapter 5 describes sustainable
development impact categories that users can ICAT’s Advisory Committee, which provides strategic
assess using this methodology, which are consistent advice to ICAT, reviewed the second draft. More
with the SDGs. Chapter 12 provides guidance on information about the development process,
monitoring progress towards the SDGs. including governance of the initiative and the
participating countries, is available on the ICAT
website.

1.8 Calculation methods, models All contributors are listed in the Contributors section
and tools for assessing impacts at the end of the document.

This document outlines a general process that

users should follow when assessing the impacts of
policies, but does not prescribe specific calculation
methods or tools that should be used. Users
should supplement the methodology with models,
calculation tools, spreadsheets or other methods to
carry out calculations.
10
https://climateactiontransparency.org/icat-toolbox/sustainable-
development
11
Listed at https://climateactiontransparency.org/icat-toolbox/
9
https://sustainabledevelopment.un.org/sdgs sustainable-development.
2 Objectives of assessing sustainable
­development impacts

This chapter provides an overview of objectives users (environmental, social and economic) that
may have in assessing the sustainable development are most relevant to national priorities and
impacts of policies. Determining the assessment stakeholders.
objectives is an important first step, since decisions
made in later chapters should be guided by the stated • Maximize positive impacts, and minimize
objectives. and mitigate negative impacts of policies
across multiple impact categories and across
Checklist of key recommendations different groups in society.

• Determine the objectives of the assessment • Ensure that policies are cost-effective and
at the beginning of the impact assessment that limited resources are invested efficiently.
process
• Align policies with national and international
laws and principles on sustainable
Assessing the impacts of policies is a key step development, climate change and human
towards developing effective sustainable rights, and with national laws and regulations
development strategies. Impact assessment supports relating to environmental and social impact
evidence-based decision-making by enabling assessment.
policymakers and stakeholders to understand
the relationship between policies and expected
or achieved changes in various sustainable
development impact categories. 2.2 Objectives of assessing impacts
before policy implementation
It is a key recommendation to determine the
objectives of the assessment at the beginning of the • Improve policy selection, design and
impact assessment process. Examples of objectives implementation by comparing policy options
for assessing the sustainable development impacts based on their expected future impacts
of a policy are provided below. across multiple impact categories, and
understanding the impacts of different design
and implementation choices.

2.1 General objectives • Inform goal-setting by assessing the

potential contribution of policy options to
• Identify and promote policies that address national or subnational goals, such as SDGs
multiple priorities, contribute to multiple and NDCs, and understand whether planned
goals and lead to multiple benefits, such as policies are sufficient to meet goals.
improved health from reduced air pollution;
job creation; poverty reduction; climate • Report on the expected future impacts of
change mitigation; increased energy access; policies, domestically or internationally.
gender equality; and others identified in
development strategies, the SDGs, NDCs • Access financing for policies under
under the Paris Agreement, and other national consideration by demonstrating net benefits
plans to promote policy coherence and across multiple impact categories.
integrated national strategies.

• Integrate climate policy into broader

national development policy and broaden
support for climate actions by assessing and
communicating the impacts of climate actions
12 Sustainable Development Methodology

2.3 Objectives of assessing

impacts during or after policy
implementation

• Assess policy effectiveness and improve

implementation by determining whether
policies are being implemented as planned
and delivering the intended results across
multiple impact categories and across
different groups in society.

• Inform adjustments to policy design and

implementation, and decide whether to
continue current actions, enhance current
actions or implement additional actions.

• Learn from experience and share best

practices about the impacts of policies.

• Track progress towards national goals

such as NDCs and SDGs, and understand the
contribution of policies to achieving them.

• Report on the impacts of policies achieved to

date, domestically or internationally.

• Meet funder requirements to report on

sustainable development impacts of policies, if
applicable.

Users should identify the intended audience(s) of

the assessment report. Possible audiences include
policymakers, the general public, NGOs, companies,
funders, financial institutions, analysts, research
institutions and other stakeholders affected by, or
who can influence, the policy. For more information
on identifying stakeholders, refer to the ICAT
Stakeholder Participation Guide.

Subsequent chapters provide flexibility to enable

users to choose how best to assess the impacts of
policies in the context of their objectives, including
which impacts to include in the assessment
boundary, and which methods and data sources
to use. Users can follow a qualitative and/or a
quantitative assessment approach, depending on
their objectives (further explained in Chapter 3). The
appropriate level of accuracy and completeness is
likely to vary by objective. Users should assess the
impacts of policies with a sufficient level of accuracy
and completeness to meet the stated objectives of
the assessment.
3 Key concepts, steps and planning the
­assessment

This chapter introduces key concepts in the include both positive and negative impacts to enable
methodology, provides an overview of the steps decision makers to understand the full range of
involved in assessing sustainable development impacts impacts and maximize net benefits resulting from
of policies, and provides guidance on planning the policies.
assessment.

Checklist of key recommendations 3.1.2 Indicators and parameters

• Base the assessment on the principles An “indicator” is a metric that can be estimated to
of relevance, completeness, consistency, indicate the impact of a policy on a given impact
transparency and accuracy category, or can be monitored over time to enable
tracking of changes towards targeted outcomes. For
example, to measure the impact of a policy on jobs,
a key indicator is “number of people employed”.
3.1 Key concepts Indicators are what the user aims to calculate to
assess the impacts of the policy.
This section describes key concepts that are relevant
to several chapters in the methodology. It introduces Calculating the impact of a policy on a given indicator
concepts and steps that are elaborated in more may require collecting data on multiple parameters.
detail in later chapters. It is intended as an overview, “Parameters” are the data needed to calculate the
but not to provide practical guidance, which begins in value of an indicator, in cases where the indicator
Chapter 4. cannot be directly measured. In some cases,
indicators are sufficient, and additional parameters
are not necessary. For example, it may be possible to
3.1.1 Sustainable development dimensions, measure the indicator “number of people employed”
impact categories and specific impacts directly. In other cases, parameters are necessary to
measure the indicator value. For example, estimating
Impact assessment is the qualitative or quantitative household cost savings from an energy efficiency
assessment of impacts resulting from a policy. In programme requires estimating the electricity price
this methodology, sustainable development impacts and the quantity of energy consumed in the baseline
include all types of impacts across three overarching scenario and policy scenario. In this example,
“dimensions”: environmental, social and economic. “household cost savings” is the indicator, while
“electricity price” and “quantity of energy consumed”
Within each dimension are various “impact are parameters. These two parameters are not
categories”, which are types of sustainable themselves indicators of interest, but are necessary
development impacts affected by a policy, such as to calculate the value of the indicator of interest
air quality, health, jobs, poverty reduction, access (i.e. household cost savings). Whether a given metric
to energy, gender equality, biodiversity, and energy is labelled an indicator or a parameter depends
independence, among others outlined in Chapter 5. on the specific context. In the previous example,
Users choose which impact categories to include in “quantity of energy consumed” would be an indicator
the assessment in Chapter 5. rather than a parameter if the user intends to assess
the impact of the policy on energy use.
Finally, a “specific impact” is a more specific change
(within a selected impact category) that results from Figure 3.1 provides a summary of these concepts.
a policy, such as an increase in jobs in the solar In the figure, the level of detail, specificity and
photovoltaic (PV) manufacturing industry resulting disaggregation increases from the top of the figure
from a solar PV incentive policy. Users identify (dimensions) to the bottom (parameters).
specific impacts of the policy (within selected impact
categories) in Chapter 6. Users are encouraged to
14 Sustainable Development Methodology

FIGURE 3.1

Overview of sustainable development dimensions, impact categories, specific impacts,

indicators and parameters

TERM DEFINITION EXAMPLES

Environmental
An overarching category of sustainable
Dimension Social
development impacts
Economic

Jobs Gender equality

A type of sustainable development
Impact category Air quality Poverty
impact affected by a policy
Energy access Health

An increase in jobs in the solar PV

A specific change that results from a manufacturing industry resulting from a
Specific impact
policy (within a given impact category) solar PV incentive policy (specific impact
within the jobs impact category)

A metric that can be estimated to

Number of people imployed
indicate the impact of a policy on a given
Emissions of PM2.5
Indicator impact category, or monitored over time
Percentage of energy from domestic
to enable tracking of changes towards
sources
targeted outcomes

Data needed to calculate the value of an Installed capacity of solar PV

Parameter indicator, in cases where the indicator Emission factor for PM2.5
value cannot be directly measured Electricity price

3.1.3 Assessment boundary and The assessment period is the time period over which
assessment period impacts resulting from the policy are assessed.
The assessment period may differ from the policy
The assessment boundary defines the scope of the implementation period, which is the time period
assessment in terms of the range of dimensions, during which the policy is in effect. Chapters 7
impact categories and specific impacts that are and 8 provide more information on defining the
included in the assessment. The assessment assessment period.
boundary may be broader than the geographic
and sectoral boundary within which the policy is
implemented. 3.1.4 Attribution of impacts
to policies and actions
Chapter 7 provides guidance on defining the
qualitative assessment boundary. Chapter 8 provides This methodology can support users in attributing
guidance on defining the quantitative assessment sustainable development impacts to a specific policy
boundary. All specific impacts identified in Chapter 6 (or package of policies) and understanding how
should be included in the qualitative assessment effective policies are in achieving desired results,
boundary, whereas the quantitative assessment which supports the objectives listed in Chapter 2.
boundary should include all significant impacts,
where feasible.
 Part I: Introduction, objectives and key concepts 15

Attributing impacts to specific policies is difficult, 3.1.5 Tracking progress of indicators

since changes in the world are the result of many over time
factors, including (1) the policy being assessed, (2)
other policies that directly or indirectly affect the An alternative to attributing impacts to specific
same impact categories, and (3) various external policies is to track trends in overall national statistics
drivers that affect the same impact categories. or monitor indicators over time relative to historical
To overcome this challenge, it is helpful to define values, goal values, and values at the start of policy
a baseline scenario that represents what is most implementation (detailed in Chapter 12).
likely to happen in the absence of the policy being
assessed. Monitoring trends in indicators highlights changes in
the targeted outcomes of a policy, which is helpful
For example, a city may implement a green jobs in understanding whether a policy is on track.
programme and then observe that the following Monitoring key indicators is also necessary to assess
year jobs have declined. However, the fact that progress towards goals and see whether desired
jobs declined does not mean that the policy was results are being achieved. For example, to track the
unsuccessful or caused the decrease in jobs. A progress of an energy efficiency policy, a user may
correlation between a policy being implemented track electricity consumption over time from the date
and a decline in jobs is not sufficient to establish the policy was implemented and observe whether
causation. Instead, jobs may have declined because energy consumption is declining.
of a broader economic downturn. The policy may still
have been effective in increasing jobs relative to a However, tracking indicators does not explain why
baseline scenario. changes have occurred or demonstrate cause-and-
effect relationships between interventions and
Attribution of impacts is embedded in the impacts, since it does not involve defining a baseline
quantitative impact assessment method included in scenario. For example, if energy consumption
this methodology. To estimate an impact resulting declines from one year to the next, the change could
from a policy, users follow three basic steps: be the result of the energy efficiency policy or the
result of a mild winter, which reduces demand for
1. Define the baseline scenario and estimate home heating. To attribute impacts to a policy, a
baseline scenario conditions (Chapter 8). baseline scenario is needed.

2. Define the policy scenario and estimate policy Figure 3.2 illustrates the difference between
scenario conditions (Chapters 9 and 10). attributing impacts to specific policies relative to a
baseline scenario and tracking changes in indicators
3. Subtract the baseline scenario value from the over time relative to historical values. Users can
policy scenario value to estimate the impact of follow the attribution approach, the approach of
the policy (Chapters 9 and 10). tracking indicators over time, or both approaches.
Section 3.3.1 provides guidance on choosing an
Attributing impacts to policies is also part of the approach.
qualitative impact assessment method, which
involves identifying impacts through a causal chain
that illustrates the cause-and-effect relationships 3.1.6 Qualitative and quantitative
between a policy and impacts. approaches to impact assessment

In complex situations, a causal link between a Impacts can be assessed qualitatively and/or
given policy and a given result cannot always be quantitatively. Qualitative assessment involves
demonstrated with a high degree of certainty or describing the impacts of a policy in descriptive
accuracy. Users and stakeholders should exercise terms. This can be useful for concepts that are
caution in interpreting the assessment results, which harder to measure, such as quality, behaviour or
are only as reliable as the data and methods used. experiences. Quantitative assessment involves
In situations with high complexity or uncertainty, it estimating the impacts of a policy in numerical terms,
may be more appropriate to conclude that a policy using measured or estimated data.
contributes to achieving a desired outcome than to
attribute a specific change to the policy.
16 Sustainable Development Methodology

FIGURE 3.2

Tracking indicators over time versus attributing impacts to policies and actions

TRACKING INDICATORS OVER TIME ATTRIBUTING IMPACTS TO POLICIES

io
nar
ne sce
seli
INDICATOR VALUE

INDICATOR VALUE
Historical value Ba
IMPACT
OF
POLICY
Indi Poli
cato cy s
r va cen
lue ario

Goal value

YEAR 1 YEAR 2 YEAR 1 YEAR 2

Source: Adapted from WRI (2014).

These approaches are further described in assessed. The policy scenario is the same as the
Section 3.3.1. Guidance on the qualitative approach baseline scenario except that it includes the policy (or
to impact assessment is provided in Part III, and package of policies) being assessed. The difference
guidance to the quantitative approach is provided between the policy scenario and the baseline scenario
in Part IV. The quantitative approach involves first represents the impact of the policy (see Figure 3.3).
following the qualitative approach in Part III as a
precursor step to identify and prioritize impacts, The baseline scenario can be higher or lower than
before quantifying significant impacts in Part IV. the policy scenario, depending on the situation. In
the case of a policy that reduces air pollution, the
baseline scenario would be higher than the policy
3.1.7 Baseline scenario and policy scenario scenario, since emissions are lower in the policy
scenario than in the baseline scenario. In the case
A baseline scenario, or reference case against which of a policy that increases jobs, the baseline scenario
change is assessed, needs to be established to would be lower than the policy scenario, since the
attribute impacts to a policy. The baseline scenario number of jobs is greater in the policy scenario than
represents the events or conditions most likely to in the baseline scenario.
occur in the absence of the policy being assessed.
The baseline scenario is an assumption about Chapter 8 provides guidance on developing the
conditions that would exist over the assessment baseline scenario. Chapters 9 and 10 provide
period if the policy were not implemented. guidance on developing the policy scenario, either
These conditions include other policies that are ex-ante or ex-post.
implemented, as well as external drivers and
market forces that affect the impact category being
assessed. 3.1.8 Ex-ante and ex-post assessment

In contrast to the baseline scenario, the policy An assessment is classified as either ex-ante or
scenario represents the events or conditions most ex-post depending on whether it is prospective
likely to occur in the presence of the policy being (forward-looking) or retrospective (backward-
 Part I: Introduction, objectives and key concepts 17

FIGURE 3.3

Baseline and policy scenarios

nario
ne sce
Baseli
INDICATOR VALUE

IMPACT
OF
POLICY
OR
Historical
ACTION
values
Policy
scenar
io

DATE OF ASSESSMENT TIME

Source: Adapted from WRI (2014).

looking). Ex-ante assessment is the process of In several steps throughout the methodology, users
assessing expected future impacts of a policy. Ex- should collect disaggregated data and assess impacts
post assessment is the process of assessing historical separately for different groups, where relevant,
impacts of a policy. Ex-ante assessment can be in addition to assessing total impacts based on
carried out before or during policy implementation, aggregated data. For example, users could collect
while ex-post assessment can be carried out during data on socioeconomic status separately for women
or after policy implementation. and men.

3.1.9 Distributional impacts

3.2 Overview of steps
In many cases, it may be important to separately
assess the impacts of policies on different groups in This document is organized according to the
society, such as men and women, people of different steps a user follows in assessing the sustainable
income groups, people of different racial or ethnic development impacts of a policy (see Figure 1.1).
groups, people of different education levels, people Users can skip certain parts or chapters depending
from different geographic regions, and people in on their objectives, when the methodology is
urban versus rural locations. This allows users to applied and the methodological approach chosen.
understand distributional impacts on different Users who only want to assess impacts qualitatively
groups, manage trade-offs in cases where policies without quantifying any impacts can skip Part IV.
have positive impacts on some groups and negative Within Part IV, users assessing impacts ex-post
impacts on other groups, and avoid situations where but not ex-ante should skip Chapter 9, while users
policies would be discriminatory or have adverse assessing impacts ex-ante but not ex-post should
effects on disadvantaged or vulnerable populations. skip Chapter 10. Users who only want to track
For example, a tax policy may be regressive by indicators over time without assessing impacts either
imposing more costs on poorer people than on qualitatively or quantitatively can skip Part III, IV and
wealthier people. VI. Figure 3.4 provides an example of following the
steps for a solar PV incentive policy.
18 Sustainable Development Methodology

FIGURE 3.4

Example of following the steps for a solar PV incentive policy

Part I: Understand background and define objectives

Determine the objectives of the assessment (Chapter 2): The primary objective is to improve the design
of the policy and maximize its net benefits by understanding the environmental, social and economic impacts
of various policy design options.

Part II: Defining the assessment

Clearly describe the policy to be assessed (Chapter 4): The policy is the Grid-Connected Solar Rooftop
Programme (elaborated in Table 4.1)
Choose which impact categories and indicators to assess (Chapter 5): The following impact categories
are relevant and significant, and will be assessed: climate change mitigation; air quality and health; waste;
renewable energy generation; access to clean, affordable and reliable energy; capacity, skills and knowledge
development; quality and safety of working conditions; jobs; income; new business opportunities; energy
independence (see Table 5.2). Indicators for each impact category are selected.

Part III: Qualitative approach to impact assessment

Identify specific impacts of the policy within chosen impact categories (Chapter 6): Many specific
impacts are identified, such as reduced GHG emissions and air pollution from fossil fuel–based power plants;
increased access to clean, affordable and reliable electricity; increased jobs and business opportunities in the
solar manufacturing, installation, operation and maintenance sectors; decreased business opportunities in
the fossil fuel extraction and related sectors; and increased energy independence from reduced imports of
fossil fuels (see Table 6.3).
Qualitatively assess each specific impact (Chapter 7): Each specific impact is assessed based on its
likelihood of occurring, its expected magnitude (major, moderate or minor), and the nature of the change
(positive or negative) (see Table 7.5).

Part IV: Quantitative approach to impact assessment

Estimate baseline values for impacts included in the quantitative assessment boundary
(Chapter 8): For each indicator in the quantitative assessment (e.g. number of jobs), baseline scenario values
(the conditions most likely to occur in the absence of the policy) are estimated, such as 100,000 jobs in the
solar sector per year over the assessment period (2020–2030).
Estimate policy scenario values and policy impact (ex-ante) (Chapter 9): For each indicator in the
assessment (e.g. number of jobs), policy scenario values (i.e. the conditions most likely to occur in the
presence of the policy) are estimated, such as 200,000 jobs in the solar sector per year over the assessment
period (2020–2030). The policy impact is estimated by subtracting baseline values from policy scenario values
(in this case, a forecasted increase of 100,000 jobs per year resulting from the policy).
Estimate policy scenario values and policy impact (ex-post) (Chapter 10): After the policy is
implemented, the baseline scenario is revised for each indicator (e.g. there would have been 125,000 jobs
per year without the policy in place, due to costs of solar panels falling more than expected, leading to
higher demand for solar electricity). The actual number of jobs with the policy in place is determined (such as
250,000 jobs in the solar sector), and the policy impact is estimated by subtracting baseline values from policy
scenario values (e.g. an increase of 125,000 jobs per year resulting from the policy). (See Table 9.1.)
Assess uncertainty (Chapter 11): Uncertainty and sensitivity of the results are assessed, resulting in an
uncertainty range or description (e.g. the policy is expected to create 100,000 ± 25,000 jobs per year).
 Part I: Introduction, objectives and key concepts 19

FIGURE 3.4, continued

Example of following the steps for a solar PV incentive policy

Part V: Monitoring and reporting

Monitor the performance of indicators over time (Chapter 12): Various indicators (such as the number
of jobs) are tracked over time relative to historical values, goal values, and values at the start of policy
implementation.
Report the results and methodology used (Chapter 13): The results (such as the estimated impact of the
solar PV incentive policy on the various impact categories included in the assessment) are reported, and the
assumptions, methods and data sources used are transparently documented.

Part VI: Decision-making and using results

Interpret results, evaluate synergies and trade-offs, and decide which policies to implement
(Chapter 14): Cost-effectiveness analysis is used to determine which policy design option delivers the
greatest positive impact on a given impact category (e.g. jobs) for a given level of resources. Cost–benefit
analysis and multicriteria analysis are used to determine which policy design option delivers the greatest net
benefits across multiple impact categories. Based on the results, a recommendation is made on which policy
design option to implement.

3.3 Planning the assessment impact assessment, and (3) tracking progress of
indicators over time:
Users should review this methodology and plan in
advance the steps, responsibilities and resources • Qualitative impact assessment involves
needed to meet their objectives for assessing describing and characterizing the expected
sustainable development impacts. The time and or achieved impacts of a policy on selected
human resources required to carry out an impact impact categories using qualitative
assessment depend on a variety of factors, such classifications of likelihood, magnitude and
as the complexity of the policy being assessed, the the nature of the change (positive or negative).
range of sustainable development impact categories This approach is covered in Part III.
included in the assessment, the extent of data
collection needed and whether relevant data have • Quantitative impact assessment involves
already been collected, whether analysis related to estimating the quantitative impacts of a policy
the policy has previously been done, and the desired on selected impact categories relative to a
level of accuracy and completeness needed to meet baseline scenario. Quantification includes
the user’s objectives. Users should document their qualitative impact assessment as a preliminary
plans for the assessment. step. This approach is covered in Part IV.

• Tracking progress of indicators over time

3.3.1 Choosing an overarching approach involves monitoring trends in key indicators
to applying the methodology over time relative to historical values, goal
values and values at the start of policy
Users should decide how to apply the methodology implementation. This approach is covered in
in the context of their objectives and available Part V.
resources. The methodology contains steps related
to (1) qualitative impact assessment, (2) quantitative Each approach is useful for different purposes. The
recommended approach is to follow all chapters and
therefore use all three approaches in combination.
20 Sustainable Development Methodology

This involves qualitatively assessing all identified To ensure proper interpretation of the results, users
impacts, and then quantifying the subset of impacts should report whether the assessment consists of a
that are determined to be significant and feasible qualitative impact assessment, a quantitative impact
to quantify. However, users can choose to follow assessment, and/or tracking progress of indicators
only certain steps and approaches, depending on over time.
their objectives. Table 3.1 outlines advantages and
disadvantages of each approach. Box 3.1 provides
more information on choosing an approach based
on the assessment objectives.

TABLE 3.1

Advantages and disadvantages of different approaches for applying the methodology

Approach Advantages Disadvantages

Assess impacts • Gives an understanding of expected • Does not enable a quantified estimate of
qualitatively only impacts in descriptive rather than the impacts of a policy, which limits the
numerical terms range of objectives the assessment can
• Easier; simpler; and requires less time, meet
resources and capacity • Risk of oversimplification or limited
understanding of relevant impact drivers

Assess impacts • Enables more robust and accurate • Increased time, cost, data and capacity
quantitatively (which understanding of the impacts of policies needs, depending on approach taken
includes qualitative • Enables the best understanding of (simpler to more complex)
assessment as a first trade-offs between impact categories
step)
• Meets wider set of objectives (related to
understanding policy impact)
• Meets widest set of stakeholder needs

Track progress of • Enables understanding of whether • Does not enable an estimate of

indicators over time only indicators of interest are moving in the “impact” of a policy, because changes in
right direction in relation to goal levels, indicators are not attributed to individual
such as SDGs policies, which limits the range of
• Easier; simpler; and requires less objectives the assessment can meet
resources and capacity
• In some cases, sufficient to meet
objectives, such as tracking progress
towards national goals

Use all three • Meets widest set of objectives (related • Increased time, cost, data and capacity
approaches in to understanding policy impact and needs, depending on approach taken
combination (the default tracking progress of indicators over time) (simpler to more complex)
approach presented in • Provides flexibility to use the most
the methodology) appropriate method for various impacts
 Part I: Introduction, objectives and key concepts 21

BOX 3.1

Choosing an approach based on objectives

If the user’s objective is to understand policy impacts to meet a variety of objectives – such as informing policy design,
improving policy implementation, evaluating policy effectiveness, reporting on policy impacts and attracting finance based
on policy impacts – the user should assess impacts qualitatively and/or quantitatively, rather than only tracking indicators
over time. Such users should also track progress of indicators over time, where relevant.
Whether to follow a qualitative or quantitative approach (or both) should be guided by the nature of impacts being
assessed, and the user’s objectives, capacity and resources. For some types of impacts, quantitative analysis will yield the
most meaningful results (for impacts best measured in numerical terms), whereas qualitative assessment may be most
appropriate for impacts that are not easily measured numerically or for which qualitative information provides more
meaningful results.
Some objectives may be achieved with a qualitative approach, such as gaining an understanding of a wide variety of impacts
in a short amount of time to guide decision-making. Other objectives may require a more rigorous quantitative approach,
such as attracting public or private financing to implement an intervention and achieve specific results. The quantitative
approach to impact assessment better supports several objectives, but generally requires more time and resources. The
qualitative approach is less resource-intensive, but may not fully meet all of a user’s objectives. In cases where quantification
would yield the most meaningful results, users should quantify significant impacts of the policy, where feasible, and
qualitatively assess impacts where quantification is not feasible.
If the objective is to track national or subnational progress over time, track progress towards goals such as SDGs, or track
progress of indicators to understand whether the policy is being implemented as planned, users should track progress of
indicators over time. Such users can also assess impacts qualitatively and/or quantitatively. Monitoring indicators is useful
for understanding overall progress over time and progress towards meeting goals (such as SDGs or various national goals).
It also enables an understanding of whether indicators are moving in the right direction in relation to goal levels (if relevant).
However, it does not allow changes in indicators to be attributed to individual policies.

3.3.2 Choosing a desired level of accuracy qualitative or quantitative approach (or both), and
based on objectives what types of data and methods to use. The range of
approaches is summarized in Table 3.2 and further
This methodology provides a range of approaches described in the following sections.
to allow users to manage trade-offs between the
accuracy of the results and the resources, time and Data constraints may limit the scope of the
data needed to complete the assessment, based assessment and therefore the objectives served
on individual objectives. Some objectives require by the assessment results. Users should consider
more detailed assessments that yield more accurate data availability when determining the assessment
results (to demonstrate that a specific change in a objectives and scope. Given the uncertainties
sustainable development outcome is attributable to resulting from the range of data and methods
a specific policy, with a high level of certainty), while that can be used, assessment results should be
other objectives may be achieved with simplified interpreted as “estimates” of the impact of policies.
assessments that yield less accurate results (to show
that a policy contributes to improving a sustainable
development outcome, but with less certainty
around the magnitude of the impact).

Users should choose methods that are sufficiently

accurate to meet the stated objectives of the
assessment and ensure that the resulting claims
are appropriate – for example, claims that a policy
contributes to achieving an outcome or that a
certain outcome can be attributed to a policy. Two
key choices in this regard are whether to apply a
22 Sustainable Development Methodology

TABLE 3.2

Advantages and disadvantages of different approaches for applying the methodology

Intermediate More robust results;

Methodological Less robust results; results; intermediate more resources
options fewer resources required resources required required

Number of impact Relatively few impact Multiple impact All relevant and
categories to assess categories are assessed categories are significant impact
assessed, but not all categories are assessed
relevant and significant
impact categories are
assessed

Qualitative versus Most or all impact categories Some impact Most impact categories
quantitative impact are assessed qualitatively; only categories are assessed are assessed
assessment the most significant impact qualitatively; some are quantitatively; impacts
categories, or no impact assessed quantitatively where quantification is
categories, are assessed not feasible are assessed
quantitatively qualitatively

Data Data are largely sourced from Mix of data sources Data are locally
international defaults or proxy with varying quality is specific; new values are
data from other regions; data used estimated specific to
quality is relatively low the local context; data
quality is relatively high

Methods Simplified calculation methods Mix of methods is used More sophisticated

and assumptions are used calculation methods and
assumptions are used

3.3.3 Planning data collection In some cases, the availability of certain data and
the lack of other data will dictate which methods
Collecting data is a key step in the assessment can be used. Table 3.3 outlines different options for
process. Data needs will vary, depending on the applying the methodology, depending on the range
impact categories selected for the assessment in of data available. In cases of low data availability,
Chapter 5 and the methods used to quantitatively users should consider whether new data collection
or qualitatively assess impacts in Chapters 6–11. is possible to allow a more rigorous assessment.
Users should identify data needs and collect the To guide the types of data that should be collected,
necessary data as early as possible in the process. users should consider the intended level of accuracy
Where possible, data collection should begin before and completeness of the assessment, based on
policy implementation to demonstrate before and the objectives of the assessment, and on the time,
after trends in key indicators, especially for ex-post resources and capacity available for the assessment.
assessments. Chapter 12 provides further guidance
on collecting data and preparing a monitoring plan.
 Part I: Introduction, objectives and key concepts 23

TABLE 3.3

Range of approaches for applying the methodology, based on data availability

Chapter Approaches requiring less data Approaches requiring more data

Chapter 2: Objectives • Limit the objectives to those that can be • Choose from a wider range of
achieved with fewer data requirements. objectives, including those for which
a more accurate and complete
assessment is needed.

Chapter 5: Choosing • Include a more limited set of impact • Include a wider set of impact
which impact categories categories and indicators in the assessment. categories and indicators in the
and indicators to assess assessment.

Chapter 6: Identifying • Use simplified or subjective methods to • Use evidence-based and objective
specific impacts within identify specific impacts. methods to identify specific impacts.
each impact category

Chapter 7: Qualitatively • Use simplified or subjective methods to • Use evidence-based and objective
assessing impacts qualitatively assess impacts. methods to qualitatively assess
impacts.

Chapter 8: Estimating • Quantify fewer impacts and indicators; assess • Quantify a wider set of impacts and
the baseline more impacts and indicators qualitatively. indicators.
• Use baseline values from published data • Estimate new baseline values specific
sources or proxy data from other regions. to the local context.
• Use simplified baseline assumptions and • Use more sophisticated baseline
methods. assumptions and methods.
• Include fewer drivers in the baseline scenario. • Include more drivers in the baseline
scenario.

Chapter 9: Estimating • Use policy scenario values from published • Estimate new policy scenario values
impacts ex-ante data sources or proxy data from other specific to the local context.
regions. • Use locally specific data.
• Use international default values or national- • Use more sophisticated assumptions
average data. and methods.
• Use simplified assumptions and methods.

Chapter 10: Estimating • Use international default values or national- • Use locally specific data.
impacts ex-post average data. • Use more sophisticated calculation
• Use simplified calculation methods. methods.

Chapter 11: Assessing • Use qualitative uncertainty methods. • Use quantitative uncertainty methods.
uncertainty • Use sensitivity analysis for a more limited set • Use sensitivity analysis for a wider set
of indicators. of indicators.

Chapter 12: Monitoring • Monitor a more limited set of indicators. • Monitor a wider set of indicators.
performance over time • Monitor indicators less frequently. • Monitor indicators more frequently.

Chapter 13: Reporting • Report on all assumptions, data sources, • Report on all assumptions, data
methods and limitations to ensure sources, methods and limitations to
transparency. ensure transparency.
• Ensure that the uncertainty of the results is
communicated clearly, given data limitations.

Chapter 14: Evaluating • Use less data-intensive evaluation methods, • Use a wider set of evaluation methods,
synergies and trade- such as CEA and MCA, rather than CBA. such as CEA, CBA and MCA.
offs, and using results • Apply these methods to a more limited set of • Apply these methods to a wider set of
impact categories and indicators. impact categories and indicators.

Abbreviations: CBA, cost–benefit analysis; CEA, cost-effectiveness analysis; MCA, multi-criteria analysis
24 Sustainable Development Methodology

3.3.4 Planning stakeholder participation Before beginning the assessment process, users
should consider how stakeholder participation
Stakeholder participation is recommended in many can support their objectives, and include
steps throughout the methodology. It can strengthen relevant activities and associated resources
the impact assessment, and the contribution of in their assessment plans. It may be helpful
policies to sustainable development in many ways, to combine stakeholder participation for
including by: sustainable development impact assessment with
other participatory processes involving similar
• providing a mechanism through which stakeholders for the same or related policies, such as
people who are likely to be affected by, or those being conducted for assessment of GHG and
can influence, a policy have an opportunity transformational impacts, and for technical review.
to raise issues and have these issues
considered before, during and after policy It is important to conform with national legal
implementation requirements and norms for stakeholder
participation in public policies. Requirements
• raising awareness and enabling better of specific donors, and of international treaties,
understanding of complex issues for all conventions and other instruments that the country
parties involved, building their capacity to is party to should also be met. These are likely
contribute effectively to include requirements for disclosure, impact
assessments and consultations. They may include
• building trust, collaboration, shared specific requirements for certain stakeholder
ownership and support for policies among groups (e.g. United Nations Declaration on the
stakeholder groups, leading to less conflict Rights of Indigenous Peoples, International Labour
and easier implementation Organization Convention 169) or specific types of
policies (e.g. UNFCCC guidance on safeguards for
• addressing stakeholder perceptions of risks activities that reduce emissions from deforestation
and impacts, and helping to develop measures and degradation in developing countries).
to reduce negative impacts and increase
benefits for all stakeholder groups, including During the planning phase, users should identify
the most vulnerable stakeholder groups that may be affected by, or may
influence, the policy. Appropriate approaches should
• increasing the credibility, accuracy and be identified to engage with stakeholder groups,
comprehensiveness of the assessment including through their legitimate representatives.
by drawing on diverse expert, local and Effective stakeholder participation could be
traditional knowledge and practices – for facilitated by establishing a multi-stakeholder
example, to provide inputs on data sources, working group or advisory body consisting of
methods and assumptions stakeholders and experts with relevant and diverse
knowledge and experience. Such a group may advise
• increasing transparency, accountability, and potentially contribute to decision-making; this
legitimacy and respect for stakeholders’ rights will ensure that stakeholder interests are reflected
in design, implementation and assessment of
• enabling enhanced ambition and finance by policies, including on stakeholder participation in
strengthening the effectiveness of policies and the assessment of sustainable development impacts
credibility of reporting. of a particular policy. It is also important to ensure
that stakeholders have access to a grievance redress
Various sections throughout this methodology mechanism to protect their rights related to the
explain where stakeholder participation is impacts of the policy.
recommended – for example, in choosing which
impact categories to assess (Chapter 5), identifying Refer to the ICAT Stakeholder Participation Guide for
specific impacts within each impact category more information, such as how to plan effective
(Chapter 6), qualitatively assessing impacts stakeholder participation (Chapter 4), identify and
(Chapter 7), monitoring performance over time analyse different stakeholder groups (Chapter 5),
(Chapter 12), reporting (Chapter 13), and making establish multi-stakeholder bodies (Chapter 6),
decisions, evaluating trade-offs and interpreting provide information (Chapter 7), design and conduct
results (Chapter 14). consultations (Chapter 8), and establish grievance
redress mechanisms (Chapter 9). Appendix B of this
document summarizes the steps in this methodology
 Part I: Introduction, objectives and key concepts 25

where stakeholder participation is recommended document any changes to the data sources,
and provides specific references to relevant guidance assessment boundary, methods or any other
in the Stakeholder Participation Guide. relevant factors in the time series.

• Transparency. Provide clear and complete

3.3.5 Planning technical review (if relevant) information for stakeholders to assess
the credibility and reliability of the results.
Before beginning the assessment process, users Document all relevant methods, data sources,
should consider whether technical review of the calculations, assumptions and uncertainties,
assessment report will be pursued. The technical as well as the processes, procedures and
review process emphasizes learning and continual limitations of the assessment, in a clear,
improvement, and can help users identify areas for factual, neutral and understandable manner.
improving future impact assessments. Technical The information should be sufficient to enable
review can also provide confidence that the impacts a party external to the assessment process
of policies have been estimated and reported to derive the same results if provided with
according to ICAT key recommendations. Refer to the the same source data. Chapter 13 provides a
ICAT Technical Review Guide for more information on list of recommended information to report to
the technical review process. ensure transparency.

• Accuracy. Ensure that the estimated impacts

are systematically neither over nor under
3.4 Assessment principles actual values, as far as can be judged, and
that uncertainties are reduced as far as
Assessment principles underpin and guide the practicable. Achieve sufficient accuracy to
impact assessment process, especially where enable users and stakeholders to make
the methodology provides flexibility. It is a key appropriate and informed decisions with
recommendation to base the assessment on the reasonable confidence about the integrity
principles of relevance, completeness, consistency, of the reported information. If accurate data
transparency and accuracy, as follows:12 for a given impact category are not currently
available, strive to improve accuracy over time
• Relevance. Ensure that the assessment as better data become available. Accuracy
appropriately reflects the sustainable should be pursued as far as possible, but,
development impacts of the policy and serves once uncertainty can no longer be practically
the decision-making needs of users and reduced, conservative estimates should
stakeholders – both internal and external to be used. Box 3.2 provides guidance on
the reporting entity. Applying the principle conservativeness.
of relevance depends on the objectives of
the assessment, broader policy objectives, In addition to the principles above, users should
national circumstances and stakeholder follow the principle of comparability if it is relevant
priorities. This principle should be applied, to the assessment objectives – for example, if the
for example, when choosing which impact objective is to compare multiple policies based
categories to assess in Chapter 5. on their sustainable development impacts, or to
aggregate the results of multiple impact assessments
• Completeness. Include all significant and compare the collective impacts with national
impacts – both positive and negative – in the goals (described further in Box 3.3).
assessment boundary. Document and justify
any specific exclusions. This principle should • Comparability. Ensure common methods,
be applied when identifying impact categories data sources, assumptions and reporting
and specific impacts in Chapters 5 and 6. formats, such that the estimated impacts of
multiple policies can be compared.
• Consistency. Use consistent assessment
approaches, data-collection methods and
calculation methods to allow meaningful
performance tracking over time. Transparently

12
Adapted from WRI (2014).
26 Sustainable Development Methodology

BOX 3.2

Conservativeness

Conservative values and assumptions are more likely to overestimate negative impacts or underestimate positive impacts
resulting from a policy. Whether to use conservative estimates and how conservative to be depends on the objectives
and the intended use of the results. For some objectives, accuracy should be prioritized over conservativeness, to obtain
unbiased results. The principle of relevance can help guide what approach to use and how conservative to be.

BOX 3.3

Applying the principle of comparability when comparing or aggregating results

Users may want to compare the estimated impacts of multiple policies – for example, to determine which has the greatest
positive impacts. Valid comparisons require that assessments have followed a consistent methodology – for example,
regarding the assessment period; the types of impact categories, impacts and indicators included in the assessment
boundary; baseline assumptions; calculation methods; and data sources. Users should exercise caution when comparing
the results of multiple assessments, since differences in reported impacts may be a result of differences in methodology
rather than real-world differences. To understand whether comparisons are valid, all methods, assumptions and data
sources used should be clearly reported, following the principle of transparency. Comparability can be more easily achieved
if a single person or organization assesses and compares multiple policies using the same methodology.
Users may also want to aggregate the impacts of multiple policies – for example, to compare the collective impact of
several policies in relation to a national goal. Users should likewise exercise caution when aggregating the results if different
methods have been used and if there are potential overlaps or interactions between the policies being aggregated.
Chapter 4 provides more information on policy interactions.

In practice, users may encounter trade-offs between

principles when developing an assessment. For
example, a user may find that achieving the most
complete assessment requires using less accurate
data for a portion of the assessment, which could
compromise overall accuracy. Users should
balance trade-offs between principles depending
on their objectives. Over time, as the accuracy and
completeness of data increase, the trade-off between
these principles will likely diminish.
PART II
Defining the assessment
4 Defining the policy

This chapter provides guidance on clearly defining the 4.1 Describe the policy
policy. To assess the impacts of a policy, users first to be assessed
need to understand and describe the policy that will be
assessed, decide whether to assess an individual policy To effectively carry out an impact assessment (in
or a package of related policies, and choose whether to subsequent chapters), a detailed understanding and
carry out an ex-ante or ex-post assessment. description of the policy being assessed are needed.
It is a key recommendation to clearly describe the
Checklist of key recommendations policy (or package of policies) that is being assessed.
Table 4.1 provides a checklist of recommended
• Clearly describe the policy (or package of information that should be provided to enable an
policies) that is being assessed effective assessment. Table 4.2 outlines additional
information that may be relevant, depending on the
context.

Users assessing a package of policies can apply

Table 4.1 either to the package as a whole or
separately to each policy in the package. Users who
assess a modification of an existing policy, rather
than a new policy, may define the policy to be
assessed as either the modification of the policy or
the policy as a whole, depending on the objectives.

Users who are assessing the GHG impacts and/or

transformational impacts of the policy should
describe the policy in the same way to ensure a
consistent and integrated assessment.

Table 4.1 introduces an illustrative example of a

solar PV incentive policy, which is used as a running
example throughout the methodology.

FIGURE 4.1

Overview of steps in the chapter

Decide whether to assess an

Describe the policy to be Choose ex-ante or ex-post
individual policy or a package
assessed assessment
of policies
(Section 4.1) (Section 4.3)
(Section 4.2)
 Part II: Defining the assessment 29

TABLE 4.1

Checklist of recommended information to describe the policy being assessed

Information Description Example

Title of the Policy name • Grid-Connected Solar Rooftop Programme. Throughout

policy this methodology, it is referred to as the solar PV
incentive policy.

Type of policy The type of policy, such as those • Financial incentive policy
presented in Table 1.1, or other
categories of policies that may be
more relevant

Description The specific intervention(s) carried • Financial incentives: The policy provides a financial
of specific out as part of the policy, such as subsidy of up to 30% of project/benchmark cost for
interventions the technologies, processes or rooftop solar projects. It also provides concessional loans
practices implemented to achieve to solar rooftop project developers.
the policy • Eligible technology: Grid-connected rooftop and small
solar power plants with installed capacity of 1–500 kW
• Eligible sectors: Residential (all types of residential
buildings), institutional (schools, health institutions),
social sectors (community centres, welfare homes, old
age homes, orphanages, common service centres),
commercial and industrial facilities
• Contract and payment duration: Up to 30% of the
eligible financial assistance and services charges at the
time the proposal is sanctioned; the remaining 70%
after successful commissioning of projects after sample
verification on submission of requisite claims
• National budget allocated to the policy: Approximately
$750 million
• Other enabling actions under the policy:
» Training and capacity-building of stakeholders involved
in the programme, such as government staff, utilities,
regulatory commissions, banks and workers
» Development of online portal for rooftop solar systems
development programme, and registration of partners,
approvals and project monitoring

Status of the Whether the policy is planned, • The policy has been implemented (currently in effect).
policy adopted or implemented

Date of The date the policy comes into • 1 January 2016

implementation effect (not the date that any
supporting legislation is enacted)

Date of If relevant, the date the policy • Provision of financial incentives ends on 31 December
completion (if ceases, such as the date a tax is 2022.
relevant) no longer levied or the end date of
an incentive scheme with a limited
duration (not the date that the
policy no longer has an impact)
30 Sustainable Development Methodology

TABLE 4.1, continued

Checklist of recommended information to describe the policy being assessed

Information Description Example

Implementing The entity or entities that India’s Ministry of New and Renewable Energy implements
entity or entities implement(s) the policy, including the policy. Government funds are disbursed by the ministry
the role of various local, to state agencies, financial institutions, implementing
subnational, national, international agencies and other government-approved channel
or any other entities partners – these include renewable energy service
providers, system integrators, manufacturers, vendors and
NGOs.

Objectives The intended impact(s) or The policy is intended to increase deployment of solar
and intended benefit(s) of the policy (e.g. the energy; increase access to clean energy; increase energy
impacts or purpose stated in the legislation or independence; create jobs; reduce GHG emissions; and
benefits of the regulation) create an enabling environment for investment, installation,
policy capacity-building, and research and development in the
solar energy sector.

Level of the The level of implementation, such National

policy as national level, subnational level,
city level, sector level or project
level

Geographic The jurisdiction or geographic area India

coverage where the policy is implemented
or enforced, which may be more
limited than all the jurisdictions
where the policy has an impact

Sectors The sectors or subsectors that are Energy supply (grid-connected solar PV)
targeted targeted

Other related Other policies that may interact The Government of India targets installation of
policies with the policy being assessed 100,000 MW of solar power by 2022, of which 40,000 MW
is to be achieved through rooftop solar power plants
through the solar PV incentive policy.

Source: Adapted from WRI (2014). Example adapted from India’s Ministry of New and Renewable Energy.

TABLE 4.2

Checklist of additional information that may be relevant to describe the policy being assessed

Information Description Example

Relevant SDGs SDGs the policy focuses The policy is focused primarily on SDG 3 (Good health and well-
on or contributes to being), SDG 7 (Affordable and clean energy), SDG 8 (Decent
work and economic growth), SDG 9 (Industry, innovation and
infrastructure), SDG 11 (Sustainable cities and communities),
SDG 12 (Responsible consumption and production) and SDG 13
(Climate action), while also contributing to other SDGs.
 Part II: Defining the assessment 31

TABLE 4.2, continued

Checklist of additional information that may be relevant to describe the policy being assessed

Information Description Example

Specific intended Target level of key The policy aims to install 40,000 MW of rooftop solar PV by
targets, such as indicators, if applicable 2022. The policy will lead to increased solar power generation in
intended level of the country, contributing to greater energy independence, and
indicators increased jobs in the solar PV installation and maintenance sectors.
Solar energy will also provide quick alternative power during any
severe climate changes.

Title of The name(s) of National renewable energy law

establishing legislation or regulations
legislation, authorizing or
regulations or establishing the policy
other founding (or other founding
documents documents, if there is
no legislative basis)

Monitoring, References to any Monitoring and evaluation studies of the policy will be carried out
reporting and monitoring, reporting during the implementation period, as follows:
verification and verification • At the primary level of monitoring, channel partners are
procedures procedures associated responsible for monitoring parameters such as end-use
with implementing the verification and compliance. They are also responsible for
policy compiling statistical information, such as number of companies
involved in the installation.
• National monitors would be involved, for data on number of
companies and employees active within the sector.
• National monitors, consultants, institutions, civil society groups,
corporations with relevant experience, and other government
organizations would be involved, for ground verification/
performance evaluation on a random sample basis.
• Electricity generation data should be available at the beneficiary
level. However, for projects above 5 kW, the system providers
would also make generation data available to the government at
specified intervals.
• For projects 50 kWp and above, 100% field inspection is required.

Enforcement Any enforcement or If evidence is presented that the applicant’s information is incorrect,
mechanisms compliance procedures, distributed funds will be paid back.
such as penalties for
non-compliance

Reference Information to allow For more information, see: http://mnre.gov.in/solar/schemes/

to relevant practitioners and other
documents interested parties to
access any guidance
documents related to
the policy (e.g. through
websites)
32 Sustainable Development Methodology

TABLE 4.2, continued

Checklist of additional information that may be relevant to describe the policy being assessed

Information Description Example

Broader context Broader context for The current energy mix mainly consists of imported fossil fuels.
or significance of understanding the Coal remains a dominant source of power generation in India. BMI
the policy policy Research forecasted in 2017 that coal will contribute 66% to India’s
power generation mix in 2025, and electricity generation from coal
will increase by 5.8% between 2016 and 2025. In 2000, 67% of
emissions in India were from energy generation and use.
India plans a rapid increase in the renewable energy share in the
national electricity generation mix, including plans to install 175 GW
of renewable generation capacity by 2022. Solar is projected to
contribute 100 GW of installed capacity by 2022, from the current
4 GW. Recent auctions have resulted in record low tariffs of
Rs 3 (US$ 0.0446) per kWh.
Rooftop solar has significant potential to contribute to national
energy supply. Rooftop solar installed capacity reached 525 MW
in 2015. This accounts for less than 10% of the installed utility-
scale solar capacity and a very small portion of the total power
consumption in the country. The government’s target of 40 GW of
solar rooftop capacity by 2022 has injected increased ambition into
the sector.

Key stakeholders Key stakeholder groups Households, institutions (schools, health institutions), businesses,
affected by the policy project developers, workers, utilities, banks, energy access
programmes, women’s organizations and cooperatives, micro-
credit institutions, and others

Other relevant Any other relevant Various implementation models are possible under the policy:
information information • solar installations owned and operated by consumers
• solar rooftop facility owned by consumers but operated and
maintained by a third party
• solar installations owned, operated and maintained by a third
party
• solar lease model, with sale of electricity to the grid
• solar installations owned by the utility or distribution company.

Source: Adapted from WRI (2014). Example adapted from India’s Ministry of New and Renewable Energy.
Abbreviations: kWh, kilowatt-hour; kWp, kilowatt peak

4.2 Decide whether to assess an In subsequent chapters, users follow the same
individual policy or a package of general steps and requirements, whether they
policies choose to assess an individual policy or a package
of related policies. Depending on the choice, the
If multiple policies are being developed or impacts estimated in later chapters will either apply
implemented in the same time frame, users can to the individual policy assessed or to the package of
assess the policies either individually or as a package. policies assessed.
When making this decision, users should consider
the assessment objectives, the feasibility of assessing Users who are assessing the GHG impacts and/or
impacts individually or as a package, and the degree transformational impacts of a policy, following
of interaction between the policies. other ICAT methodologies, should define the policy
 Part II: Defining the assessment 33

or policy package in the same way to ensure a a package of related policies. Assessing a broader
consistent and integrated assessment, or explain package of policies may help to avoid possible
why there are differences in how the policy package negative or unintended impacts beyond the scope of
is defined across the assessments. a single policy. At the end of the assessment, users
should also consider potential trade-offs between
impact categories, in Chapter 14.
4.2.1 Overview of policy interactions
The relationship between policies will likely differ
Policies can either be independent of each other by sustainable development impact category, such
or interact with each other. Policies interact if they as air quality, health, jobs or poverty reduction
produce total impacts, when implemented together, (further described in Chapter 5). Users should
that differ from the sum of the individual impacts consider a range of relevant impact categories when
had they been implemented separately. Table 4.3 deciding whether to assess an individual policy or
and Figure 4.2 provide an overview of four possible a package of policies. Users should consider the
relationships between policies. primary objectives of the policy when determining
which impact categories to include in the analysis
Given the interrelated nature of the SDGs, multiple of policy interactions. For example, if the primary
policies are likely to be interrelated in their impacts objective of the policy is GHG mitigation, the user
on sustainable development impact categories, and should consider analysing policy interactions from
to have potential synergies and trade-offs. Some the perspective of GHG emissions, rather than
policies may be in conflict with one another, while considering all other sustainable development
others may work together to achieve sustainable impact categories. However, in this case, other
development outcomes. Users should consider relevant sustainable development impact categories
possible synergies and trade-offs between policies should still be included in the assessment in later
when deciding whether to assess a single policy or chapters.

TABLE 4.3

Types of relationships between policies

Type Description

Independent Multiple policies do not interact with each other. The combined effect of implementing the policies
together is equal to the sum of the individual effects of implementing them separately.

Overlapping Multiple policies interact, and their combined effect is less than the sum of the individual effects
if implemented separately. This category includes policies that have the same or complementary
goals (such as national and subnational energy efficiency standards), as well as counteracting
policies that have different or opposing goals (such as a fuel tax and a fuel subsidy).

Reinforcing Multiple policies interact, and their combined effect is greater than the sum of their individual
effects if implemented separately.

Overlapping and Multiple policies interact, and have both overlapping and reinforcing interactions. The combined
reinforcing effect may be greater or less than the sum of their individual effects if implemented separately.

Source: WRI (2014), adapted from Boonekamp (2006).

34 Sustainable Development Methodology

FIGURE 4.2

Types of relationships between policies

INDEPENDENT OVERLAPPING
TOTAL IMPACT = 300 TOTAL IMPACT = 250

Policy A: Policy B: Policy A: Policy B:

impact = 100 impact = 200 impact = 100 impact = 200

Overlap
= 50

REINFORCING OVERLAPPING AND REINFORCING

TOTAL IMPACT = 400 TOTAL IMPACT = 350

Policy A: Policy B: Policy A: Policy B:

impact = 100 impact = 200 impact = 100 impact = 200

Overlap
= 50

REINFORCING = 100 REINFORCING = 100

Source: Adapted from WRI (2014).

4.2.2 Choosing whether to assess an • step 2 – apply criteria to determine whether

individual policy or package of policies to assess an individual policy or a package of
policies.
This section outlines a qualitative process to
understand the expected relationship between Step 1: Characterize the type and degree
policies under consideration, when deciding whether of interaction between the policies under
to assess an individual policy or a package of policies. consideration
The most robust approach is to qualitatively assess Potentially interacting policies can be identified by
the extent of policy interactions at this stage, but this identifying activities targeted by the policy, then
is not necessary if it is not feasible. identifying other policies that target the same
activities. Once these are identified, users should
To assess the extent of policy interactions when assess the relationship between the policies
deciding whether to assess an individual policy or a (independent, overlapping or reinforcing) and
package of policies, users should follow these steps: the degree of interaction (major, moderate or
minor). Relationships between the same policies
• step 1 – characterize the type and degree may be overlapping for some impact categories
of interaction between the policies under and reinforcing or independent for other impact
consideration categories. The assessment of interaction should
be based on expert judgment, published studies
of similar combinations of policies or consultations
 Part II: Defining the assessment 35

with relevant experts. The assessment should be user can assess an individual policy (since a package
limited to a preliminary qualitative assessment at is not feasible) but acknowledge in a disclaimer that
this stage, rather than a more detailed qualitative any subsequent aggregation of the results from
or quantitative assessment, as described in later individual assessments would be inaccurate given
chapters. the interactions between the policies.

Step 2: Apply criteria to determine whether Users can also assess both individual policies
to assess an individual policy or a package of and packages of policies. Doing so will yield more
policies information than choosing only one option.
Where policies interact, there can be advantages Undertaking both individual assessments and
and disadvantages to assessing the interacting assessments for combinations of policies should be
policies individually or as a package (see Table 4.4). considered where the end user requires information
To help decide, users should apply the criteria in on both, resources are available to undertake
Table 4.5. In some cases, certain criteria may suggest multiple analyses and undertaking both is feasible.
assessing an individual policy, while other criteria
suggest assessing a package. Users should exercise If users choose to assess both an individual
judgment, based on the specific circumstances of the policy and a package of policies that includes the
assessment. For example, related policies may have individual policy assessed, users should define each
significant interactions (suggesting a package), but assessment separately and treat each as a discrete
it may not be feasible to model the whole package application of this methodology, to avoid confusion
(suggesting an individual assessment). In this case, a of the results.

TABLE 4.4

Advantages and disadvantages of assessing policies individually or as a package

Approach Advantages Disadvantages

Assessing • Shows the effectiveness of individual policies, which • The estimated impacts
policies decision makers may require to make decisions about from assessments of
individually which individual policies to support. individual policies cannot be
• May be simpler than assessing a package in some cases, straightforwardly summed
since the causal chain and range of impacts for a package to determine total impacts, if
may be significantly more complex. interactions are not accounted
for.

Assessing • Captures the interactions between policies in the package • Does not show the
policies as a and better reflects the total impacts of the package. effectiveness of individual
package • May be simpler than undertaking individual assessments policies.
in some cases, since it avoids the need to disaggregate • May be difficult to quantify.
the effects of individual policies.

Source: Adapted from WRI (2014).

36 Sustainable Development Methodology

TABLE 4.5

Criteria for determining whether to assess policies individually or as a package

Criteria Questions Recommendation

Objectives and Do the end users of the assessment results want to know the If “Yes”, undertake an
use of results impact of individual policies? individual assessment.

Significant Are there significant (major or moderate) interactions between If “Yes”, consider assessing a
interactions the identified policies, either overlapping or reinforcing, that will package of policies.
be difficult to estimate if policies are assessed individually?

Feasibility Is it possible (e.g. are data available) to assess a package of If “No”, undertake an individual
policies? assessment.

For ex-post assessments, is it possible to disaggregate the If “No”, consider assessing a

observed impacts of interacting policies? package of policies.

Source: Adapted from WRI (2014).

4.3 Choose ex-ante or ex-post of the policy on that indicator (ex-ante). In 2025, the
assessment user carries out an ex-post assessment of the same
policy to assess the historical impacts of the policy to
Users can carry out an ex-ante (forward-looking) date, by observing actual conditions over the policy
assessment, an ex-post (backward-looking) implementation period – that is, the ex-post policy
assessment, or a combined ex-ante and ex-post scenario – and defining a revised ex-post baseline
assessment. Choosing between ex-ante and ex-post scenario. The difference between the ex-post policy
assessment depends on the status of the policy. scenario and the ex-post baseline scenario is the
Where the policy is planned or adopted, but not yet estimated impact of the policy (ex-post).
implemented, the assessment will be ex-ante by
definition. Alternatively, where the policy has been If conditions unrelated to the policy unexpectedly
implemented, the assessment can be ex-ante, ex- change between 2020 and 2025, the ex-post baseline
post, or a combination of ex-ante and ex-post. In this scenario will differ from the ex-ante baseline
case, users should carry out an ex-post assessment scenario. For example, the ex-post and ex-ante
if the objective is to estimate the impacts of the baseline scenarios will differ if external factors
policy to date, an ex-ante assessment if the objective such as economic conditions differ from ex-ante
is to estimate the expected impacts in the future,13 forecasts made in 2020, or if significant new policies
and a combined ex-ante and ex-post assessment to are introduced. The ex-post policy scenario may
estimate both the past and future impacts. differ from the ex-ante policy scenario for the same
reasons, or if the policy is less (or more) effective in
Figure 4.3 illustrates the relationship between practice than it was expected to be. In such cases, the
ex-ante and ex-post assessment. In the figure, a ex-ante and ex-post estimates of the policy’s impact
policy comes into effect in 2020. The user carries will differ.
out an ex-ante assessment in 2020 to estimate the
expected future impacts of the policy on a given In an ex-ante assessment, the baseline scenario and
indicator through to 2030, by defining an ex-ante policy scenario are both hypothetical or forecasted,
baseline scenario and an ex-ante policy scenario. The rather than observed. In an ex-post assessment, only
difference between the ex-ante policy scenario and the baseline scenario is hypothetical, since the ex-
the ex-ante baseline scenario is the estimated impact post policy scenario can be observed.

13
An ex-ante assessment may include historical data if the policy is
already implemented, but it is still an ex-ante rather than an ex-post
assessment if the objective is to estimate future effects of the policy.
 Part II: Defining the assessment 37

FIGURE 4.3

Ex-ante and ex-post assessment

enario
bas eline sc
Ex-ante
INDICATOR VALUE

Ex-post baseline scenario IMPACT

OF POLICY
IMPACT OF POLICY OR ACTION
Ex- OR ACTION (EX-POST) (EX-ANTE)
Historical pos
t po
values (ob licy sce
ser
ved nario Ex-an
) te po
licy s
cena
rio

2020 2025 2030

Source: Adapted from WRI (2014).

5 Choosing which impact categories and
indicators to assess

This chapter outlines sustainable development impact categories, and guidance on choosing which impact
categories that users can assess and assists users in categories to assess.
determining which impact categories to assess for their
policy. In this chapter, users also identify indicators The policy being assessed is likely to have positive
for each included impact category that will be used in impacts on some impact categories and negative
subsequent chapters. impacts on others. Users should choose a
comprehensive set of impact categories that are
Checklist of key recommendations relevant to the assessment. In subsequent chapters,
users determine how the policy affects each impact
• Include all sustainable development impact category. In Chapter 14, users evaluate potential
categories in the assessment that are synergies and trade-offs between the selected
expected to be (1) relevant (based on the impact categories to inform decision-making.
objectives of the assessment, national or local
policy objectives, sustainable development
goals and priorities, local circumstances, and 5.1.1 Examples of impact categories
stakeholder priorities) and (2) significantly
affected by the policy (either positively or Table 5.1 lists examples of impact categories that
negatively) can be assessed. Users should review the list of
• Consult stakeholders when choosing which examples with their policy in mind to identify which
impact categories to assess impact categories may be relevant or significant for
their assessment. Users should first consider a wide
set of impact categories in this step, then determine
which of them are both relevant and significant in
5.1 Choose which impact categories Section 5.1.2.
to include in the assessment
The list is illustrative, rather than comprehensive or
Users can assess a wide variety of sustainable prescriptive. Users can choose a subset of impact
development impact categories across the three categories from this list or use the list as a starting
dimensions of environmental, social and economic point to prepare a list that best meets their needs.
impacts. Examples of impacts are improved health In consultation with stakeholders, users should
from reduced air pollution, job creation, poverty brainstorm to identify additional impact categories
reduction, increased energy access, and gender not included in the list that may be relevant or
equality. This section provides examples of impact significant.

FIGURE 5.1

Overview of steps in the chapter

Choose which impact categories to include in Identify indicators for each included
the assessment impact category
(Section 5.1) (Section 5.2)
 Part II: Defining the assessment 39

In Table 5.1, impact categories are organized into more than one dimension. For example, poverty and
groups to help with navigation. The names of impact jobs could be considered either social or economic
categories, and their classification into different impacts.
dimensions and groups, are suggestions and can be
adapted by users. Some impact categories blur the See Box 5.1 for an explanation of the relationship of
line between the social, economic and environmental the list of impact categories to the United Nations
dimensions, and could reasonably appear under SDGs.

TABLE 5.1

Examples of impact categories

Groups
of impact
Dimension categories Impact categories

Environmental Air • Climate change mitigation (SDG 13)

impacts • Ozone depletion
• Air quality and health impacts of air pollution (SDGs 3, 11, 12)
• Visibility
• Odours

Water • Availability of fresh water (SDG 6)

• Water quality (SDGs 6, 14)
• Biodiversity of freshwater and coastal ecosystems (SDGs 6, 14)
• Fish stocks sustainability (SDG 14)

Land • Biodiversity of terrestrial ecosystems (SDG 15)

• Land-use change, including deforestation, forest degradation and
desertification (SDG 15)
• Soil quality (SDG 2)

Waste • Waste generation and disposal (SDG 12)

• Treatment of solid waste and wastewater (SDG 6)

Other/cross- • Resilience of ecosystems to climate change (SDG 13)

cutting • Adverse effects of climate change (SDG 13)
• Energy (SDG 7)
• Depletion of non-renewable resources (SDG 12)
• Material intensity (SDG 12)
• Toxic chemicals released to air, water and soil
• Genetic diversity and fair use of genetic resources (SDGs 2, 15)
• Terrestrial and water acidification (SDG 14)
• Infrastructure damage from acid gases and acid deposition
• Loss of ecosystem services from air pollution
• Nuclear radiation
• Noise pollution
• Aesthetic impacts
40 Sustainable Development Methodology

TABLE 5.1, continued

Examples of impact categories

Groups
of impact
Dimension categories Impact categories

Social impacts Health and • Accessibility and quality of health care (SDG 3)
well-being • Hunger, nutrition and food security (SDG 2)
• Illness and death (SDG 3)
• Access to safe drinking water (SDG 6)
• Access to adequate sanitation (SDG 6)
• Access to clean, reliable and affordable energy (SDG 7)
• Access to land (SDG 2)
• Standard of living
• Quality of life and well-being (SDG 3)

Education and • Accessibility and quality of education (SDG 4)

culture • Capacity, skills and knowledge development (SDGs 4, 12)
• Climate change education, public awareness, capacity-building and research
• Preservation of local and indigenous culture and heritage (SDG 11)

Institutions and • Quality of institutions (SDG 10)

laws • Corruption, bribery and rule of law (SDG 16)
• Public participation in policymaking processes
• Access to information and public awareness (SDG 12)
• Compensation for victims of pollution
• Access to administrative and judicial remedies (SDG 16)
• Protection of environmental defenders
• Freedom of expression

Welfare and • Poverty reduction (SDG 1)

equality • Economic inequality (SDGs 8, 10)
• Equality of opportunities and equality of outcomes (SDG 10)
• Protection of poor and negatively affected communities (SDG 12)
• Removal of social disparities
• Climate justice and distribution of climate impacts on different groups
• Gender equality and empowerment of women (SDG 5)
• Racial equality
• Indigenous rights
• Youth participation and intergenerational equity
• Income of small-scale food producers (SDG 2)
• Migration and mobility of people (SDG 10)

Labour • Labour rights (SDG 8)

conditions • Quality of jobs (SDG 8)
• Fairness of wages (SDG 8)
• Quality and safety of working conditions (SDG 8)
• Freedom of association (SDG 8)
• Just transition of the workforce (SDG 8)
• Prevention of child exploitation and child labour (SDGs 8, 16)
• Prevention of forced labour and human trafficking (SDG 8)
 Part II: Defining the assessment 41

TABLE 5.1, continued

Examples of impact categories

Groups
of impact
Dimension categories Impact categories

Social impacts, Communities • City and community climate resilience (SDG 11)
continued • Mobility (SDG 11)
• Traffic congestion (SDG 11)
• Walkability of communities (SDG 11)
• Road safety (SDGs 3, 11)
• Community/rural development
• Accessibility and quality of housing (SDG 11)

Peace and • Resilience to dangerous climate change and extreme weather events (SDG 13)
security • Security (SDG 16)
• Maintaining global peace (SDG 16)

Economic Overall • Economic activity (SDG 8)

impacts economic • Economic productivity (SDGs 2, 8)
activity
• Economic diversification (SDG 8)
• Decoupling economic growth from environmental degradation (SDG 8)

Employment • Jobs (SDG 8)

• Wages (SDG 8)
• Worker productivity

Business and • New business opportunities (SDG 8)

technology • Growth of new sustainable industries (SDGs 7, 17)
• Innovation (SDGs 8, 9)
• Competitiveness of domestic industry in global markets
• Agricultural productivity and sustainability (SDG 2)
• Economic development from tourism and ecotourism (SDG 8)
• Transportation supply chains
• Infrastructure creation, improvement and depreciation

Income, prices • Income (SDG 10)

and costs • Prices of goods and services
• Costs and cost savings
• Inflation
• Market distortions (SDG 12)
• Internalization of environmental costs/externalities
• Loss and damage associated with environmental impacts (SDG 11)
• Cost of policy implementation and cost-effectiveness of policies

Trade and • Balance of payments

balance of • Balance of trade (imports and exports)
payments
• Foreign exchange
• Government budget surplus/deficit
• Energy independence, security or sovereignty
• Global economic partnership
42 Sustainable Development Methodology

BOX 5.1

Relationship to the United Nations Sustainable Development Goals

This methodology is intended to be consistent with the SDGs, to help countries assess the impacts of policies in contributing
to achieving the SDGs. The 17 SDGs, outlined in Figure 5.2, and the associated 169 targets are framed as aspirations or
desired outcomes rather than as a neutral list of impact categories. Table 5.1 adapts many of the SDG goals and targets
so that impact categories are expressed in neutral terms, to allow users to assess positive or negative impacts on each
impact category. To keep Table 5.1 relatively comprehensive, yet still concise and user-friendly, not all 169 SDG targets are
reflected in the table, and certain impact categories were merged. The SDG(s) most directly relevant to each impact category
is indicated in parentheses in the table. For some impact categories, there is no directly associated SDG, so not every
impact category indicates an associated SDG. Users should refer to the full list of SDG goals, targets and indicators for more
information when deciding which impact categories to assess.14
Other sources were also reviewed when developing the list of impact categories.15

FIGURE 5.2

The Sustainable Development Goals

NO ZERO GOOD HEALTH QUALITY GENDER CLEAN WATER

POVERTY HUNGER AND WELL-BEING EDUCATION EQUALITY AND SANITATION

AFFORDABLE AND DECENT WORK AND INDUSTRY, INNOVATION SUSTAINABLE CITIES RESPONSIBLE
CLEAN ENERGY ECONOMIC GROWTH AND INFRASTRUCTURE AND COMMUNITIES CONSUMPTION
AND PRODUCTION

CLIMATE LIFE LIFE PEACE, JUSTICE PARTNERSHIPS

ACTION BELOW WATER ON LAND AND STRONG FOR THE GOALS
INSTITUTIONS

Developed in collaboration with | [email protected] | +1.212.529.1010

For queries on usage, contact: [email protected]

5.1.2 Choosing which impact categories to

assess

Choosing which impact categories to assess is one of

the most important steps in the assessment process.
To ensure a complete and relevant assessment of
the impacts resulting from a policy, users should
choose which impact categories to assess based on
14
https://sustainabledevelopment.un.org/sdgs and their:
http://unstats.un.org/sdgs
15
These included UNFCCC, the Paris Agreement, decisions from
• significance
the Conference of the Parties to the UNFCCC, the Declaration of the
United Nations Conference on the Human Environment (Stockholm
Declaration), the Rio Declaration on Environment and Development • relevance.
(Rio Declaration), the United Nations Millennium Declaration, the
Johannesburg Declaration on Sustainable Development, and The
Future We Want.
 Part II: Defining the assessment 43

It is a key recommendation to include all sustainable impact categories to assess. For further information
development impact categories in the assessment on linkages between impact categories, see Box 5.2.
that are expected to be (1) relevant (based on the
objectives of the assessment, national or local As users proceed through subsequent chapters in
policy objectives, sustainable development goals this methodology, the decision about which impact
and priorities, local circumstances, and stakeholder categories are relevant and significant, and should
priorities) and (2) significantly affected by the be included in the assessment is likely to become
policy (either positively or negatively). It is also a clearer. For this reason, users should develop an
key recommendation to consult stakeholders when initial list of impact categories to assess in this
choosing which impact categories to assess. chapter, and then revisit the list after completing the
steps in Chapters 6 and 7. Box 5.3 provides more
The choice should be made in a principled, information on this iterative process.
transparent and participatory way, in the context of
the user’s objectives and the needs of stakeholders. Identifying significant impact categories
Selecting too few impact categories may not provide The most objective criterion for the selection of
an adequate reflection of a policy’s full impact, impact categories is significance, which involves
whereas selecting too many could make the process determining which impact categories are expected to
burdensome. Selecting only impact categories that be significantly affected by the policy, either positively
are expected to show positive impacts would lead or negatively. Users should review the list of impact
to an incomplete and biased assessment, as would categories in Table 5.1 and consider which may be
only selecting impact categories that are expected to significantly affected by the policy. For example, a
show negative impacts. solar PV incentive policy may be reasonably expected
to have significant impacts on air quality and energy
When choosing impact categories to include in the independence, and insignificant impacts on tourism
assessment, users should be aware that sustainable and waste generation. Table 5.2 provides a template,
development impact categories are linked and with an example, that can be used to assess each
interrelated. For example, gender equality and impact category.
empowerment of women is intertwined with many
other impact categories in Table 5.1, even if they are To ensure a complete assessment, users should
not explicitly focused on gender, such as ensuring consider a wide range of potential impacts, including
equal access to education, skills development, jobs, positive and negative, intended and unintended,
new business opportunities and equality of wages. short-term and long-term, and in-jurisdiction and
Therefore, it is important to consider a wide range of out-of-jurisdiction impacts. These types of impacts
potentially relevant and significant impact categories are detailed further in Chapter 6 (in Table 6.1).
that may be interconnected when choosing which

BOX 5.2

Interlinkages between sustainable development impact categories

When selecting which impact categories to assess, users should consider impact categories that are likely to be interrelated.
Examples of interrelated impact categories, often called “nexuses”, are:
• health, poverty, gender and education
• water, soil and waste
• education, health, food and water
• water, energy, food, land and climate
• infrastructure, inequality and resilience.
More information on interactions between impact categories and SDGs can be found in a number of resources.15

16
Jungcurt (2016); Melamed, Schmale and von Schneidemesser
(2016); Nilsson, Griggs and Visbeck (2016); ISC (2017); Nerini et al.
(2017)
44 Sustainable Development Methodology

BOX 5.3

Iterative process to identify relevant and significant impact categories in Chapters 5, 6 and 7

Chapters 5, 6 and 7 present a stepwise prioritization process for identifying impact categories and specific impacts of a
policy. In Chapter 5, users consider a broad array of possible impact categories (e.g. jobs) across the environmental, social
and economic dimensions, and identify which are relevant and significant to the policy being assessed. Next, in Chapter 6,
users identify specific impacts within the chosen impact categories (e.g. an increase in jobs from solar PV installation due
to the policy). In Chapter 7, users qualitatively assess these specific impacts and determine which should be quantified (in
Chapters 8–11), based on the criteria of significance and feasibility (e.g. the increase in jobs from solar PV installation is
significant and feasible to quantify).
In this process, users begin Chapter 5 by considering a long list of impact categories and end Chapter 7 with a short list of
specific impacts to be quantified. These steps are illustrated through the example of a solar PV incentive policy in Tables 5.2,
6.3 and 7.5.
The steps are iterative. For example, users may find in Chapter 6 or 7 that certain impact categories not deemed significant
in Chapter 5 are in fact significant and should be included in the assessment. Users should revisit Chapter 5 after going
through the steps in Chapters 6 and 7 to make sure that all potentially significant and relevant impact categories are
included in the assessment, as illustrated in Figure 5.3.

FIGURE 5.3

Iterative process to identify relevant and significant impact categories and specific impacts

Qualitatively assess specific

Identify relevant and Identify specific impacts within
impacts to determine which
significant impact categories selected impact categories
are significant
(Chapter 5) (Chapter 6)
(Chapter 7)

Users should rely on evidence when determining If it is not clear whether the policy is expected to
which impact categories may be significantly affected significantly affect a given impact category, or if the
by the policy, to ensure that potentially significant assessment objectives or other factors suggest that
impact categories are considered, even if they are an impact category should be included even if it
not immediately obvious. For example, a solar PV may not be significant, the most robust approach
incentive policy could increase waste generation is to include it in the assessment for further
significantly if PV panels or batteries need to be analysis in later chapters. Chapters 6 and 7 provide
replaced frequently, depending on whether these detailed guidance on identifying and assessing the
can be recycled. significance of specific impacts.

Evidence for determining the significance of impact Identifying relevant impact categories
categories may include published studies on Another criterion for the selection of impact
similar policies and impact categories in the same categories is their relevance, from the perspective of
or other jurisdictions, regulations, development users, decision makers and stakeholders. Relevance
plans, regulatory impact analyses, environmental is a more subjective criterion than significance.
impact assessments, risk assessments, economic It may be determined based on the objectives of
studies, relevant media reports, consultation with the assessment, national or local policy objectives,
experts and stakeholders, prior experience, or other sustainable development goals and priorities, local
methods. If evidence does not exist, expert judgment circumstances, and stakeholder priorities, as voiced
should be used. during stakeholder consultation processes.
 Part II: Defining the assessment 45

Applying the criterion of relevance involves a can enhance the completeness of the assessment,
policy decision by the user regarding which impact identify and address possible unintended or negative
categories are priorities. For example, a solar PV impacts early on, and increase acceptance of the final
incentive policy may be explicitly designed to reduce assessment results.
GHG emissions and reduce negative health impacts
caused by air pollutants, so both of these impact Users should identify the range of stakeholder
categories are relevant to the policy objectives. groups that may be affected by, or may influence,
Stakeholders such as workers in the energy sector the implementation of a policy and should ensure
may also be interested in how the policy will affect that legitimate representatives of these stakeholder
employment in affected regions, so the impact groups are included in the consultations. Users
category of jobs is also relevant. Users should include should recognize that stakeholder groups are not
as many relevant impact categories as possible, so homogeneous, and that age, ethnicity and gender
that the assessment properly addresses the policy’s may shape the perceptions and impacts that
objectives, and stakeholders’ priorities and concerns. policies will have on different individuals. Therefore,
Users should also consider certain impact categories efforts should be made to ensure that stakeholder
(e.g. poverty and gender equality) even if the policy engagement is as representative and inclusive as
is not explicitly designed to address them and possible. The ICAT Stakeholder Participation Guide
the impacts may not at first seem significant – for provides more information on how to identify
example, to develop safeguards against the policy stakeholders (Chapter 5), provide information
leading to negative or unintended impacts. to them (Chapter 7), and conduct consultations
(Chapter 8) to identify all significant and relevant
Ensuring comprehensiveness impact categories. Box 5.4 provides an example
Policies may have both positive and negative of identifying stakeholders for an assessment in
impacts on sustainable development. Identifying Mexico.
possible adverse impacts is important to make any
necessary adjustments to the policy and to assist Public participation is a means of ensuring
those who may be negatively affected. The list of good governance, transparency, accountability
impact categories to assess should therefore be and integrity of the sustainable development
comprehensive, including both positive and negative assessment. Adequate access to information and
impacts. Including possible adverse impacts in the opportunities to provide input, including through
list and later finding that such impacts have not effective consultations, will allow stakeholders
manifested or are insignificant is a useful way of to contribute their knowledge and experience to
demonstrating that the policy is appropriate. In the the evaluation of the sustainable development
case of a solar PV incentive policy, for example, it may impacts of policies. Local communities, indigenous
be relevant to include “electricity prices” and “access peoples, industry representatives, trade unions,
to clean, reliable and affordable energy” as impact civil society organizations (including women’s and
categories, to monitor any possible adverse impact of youth organizations) and researchers may have
the policy on electricity prices and energy access. very valuable input to offer as to what impact
categories are significant and relevant, so that
A comprehensive list should include impact users can achieve a comprehensive and balanced
categories from each of the three dimensions of assessment of sustainable development impacts. In
sustainable development (economic, social and most countries, laws require access to information
environmental). The goal of sustainable development and public participation in assessment of social and
calls for striking a balance between each of its environmental impacts of proposed interventions.
three dimensions. A policy with highly positive In the case of a solar PV incentive policy, public
environmental and economic impacts but highly consultations that are open to citizens at large,
negative social consequences would not be regarded municipal governments, professional associations
as truly sustainable. from the energy sector and public health researchers
may bring impact categories to the attention of the
Consulting stakeholders user that would otherwise have been left out.
Users should consult stakeholders to identify
which impact categories are priorities for different Reporting
stakeholder groups, and which meet the criteria Reporting which impact categories are included and
of significance, relevance and comprehensiveness. excluded is important to ensure that the sustainable
Different groups of stakeholders approach a development impact assessment is conducted in a
policy from different perspectives. By conducting transparent way, which will increase its legitimacy,
stakeholder consultations to identify impacts, users usefulness and replicability. Users should report
46 Sustainable Development Methodology

which impact categories are included and excluded impact categories in Table 5.1, as well as columns
from the assessment boundary, and justify any for users to indicate (1) whether each impact
exclusions of impact categories that may be relevant category is relevant (from the perspective of the
or significant, or identified by stakeholders. user, decision makers or stakeholders), (2) whether
the policy is significant (i.e. expected to significantly
Table 5.2 provides an example of reporting which affect each impact category) and (3) whether each
impact categories are included and excluded for impact category is included in the assessment
the example of the solar PV incentive policy. The boundary. Users should provide a brief rationale for
table can be used as a template to help decide the decision to include or exclude a given impact
which impact categories to assess and to report category and to explain the expected impacts of the
which impact categories are included in the policy on the impact category.
assessment boundary. It contains several of the

BOX 5.4

Identifying and mapping stakeholders of a sustainable development assessment in Mexico

A researcher at Aalto University assessed the sustainable development impacts of two climate actions in public buildings in
Mexico: installing PV panels and changing fluorescent lamps to LED lamps. Both actions are part of the Carbon Management
Plan of the Mexican state of Jalisco, which was developed by the Ministry of Environment and Territorial Development, in
cooperation with Carbon Trust. The office buildings of the Sub-Administration of the Ministry of Planning, Administration,
and Finance were the first to undergo the retrofit.
As part of the assessment, it was important to identify a balanced group of stakeholders to provide a comprehensive and
robust range of information and insights. To identify stakeholders to engage, the study used a rainbow diagram (Figure 5.4)
from the ICAT Stakeholder Participation Guide. The diagram helped identify and classify specific people or groups of people
that are both affected by the policy and have influence over the policy to varying levels. This helped identify key impact
categories for the assessment.

FIGURE 5.4

Rainbow diagram to identify relevant stakeholders for the assessment

MODERATE INFLUENCE

Least affected

MOST PROCURER LEAST

INFLUENCE INFLUENCE

STATE Moderately
CONGRESS
affected
PUBLIC
SERVICE
USERS
LED
Most MAINT.
LAMPS
SUPPLIERS
affected STAFF
CLIMATE CARBON RETROFITTED
INT’L PV
TRUST MINISTRY
GROUP SEPAF PANELS
STAFF SUPPLIERS

SEPAF
INSTALLATION FOSSIL
SEMADET MFG STAFF
WORKERS FL LAMPS FUTURE FUELS TECH.
GOV. OF POPULATION
SUPPLIERS SUPPLIERS
JALISCO
 Part II: Defining the assessment 47

TABLE 5.2

Example of reporting which impact categories are included in the assessment

for a solar PV incentive policy (hypothetical example only)

Included
in the Brief description
Impact assessment (rationale for determination
Dimension category Relevant? Significant? boundary? of relevance and significance)

Environmental Climate Yes Yes Yes The policy is expected to

change significantly reduce GHG
mitigation emissions by replacing fossil fuel
energy with solar energy.

Air quality, Yes Yes Yes The policy is expected to

health impacts significantly reduce air pollution
of air pollution by replacing fossil fuel energy with
solar energy.

Waste Yes Yes Yes The policy is expected to have

generation both positive and negative
and disposal impacts on waste by reducing
fossil fuel energy waste and
increasing solar energy waste
(e.g. PV panels or batteries).

Energy Yes Yes Yes The policy is expected to

significantly increase renewable
energy generation by replacing
fossil fuel energy with solar
energy.

Availability of No Yes No The policy is expected to increase

fresh water the availability of fresh water
by reducing water used by coal
power plants, but assessing
availability of fresh water is not
relevant to the assessment
objectives and was not expressed
as a priority of stakeholders.

Land-use Yes No No The policy is not expected to

change significantly affect these impact
categories in the local context.
Biodiversity Yes No No
of terrestrial
ecosystems

Soil quality Yes No No

Nuclear Yes No No
radiation
48 Sustainable Development Methodology

TABLE 5.2, continued

Example of reporting which impact categories are included in the assessment

for a solar PV incentive policy (hypothetical example only)

Included
in the Brief description
Impact assessment (rationale for determination
Dimension category Relevant? Significant? boundary? of relevance and significance)

Social Access Yes Yes Yes The policy is not expected to

to clean, increase access to energy, since
affordable and all eligible households and
reliable energy buildings are already connected
to the electricity grid, but is
expected to significantly improve
access to clean, affordable and
reliable energy.

Capacity, Yes Yes Yes The policy is expected to

skills and significantly improve training
knowledge for skilled workers in the solar
development manufacturing, installation and
maintenance sectors.

Quality and Yes Yes Yes The policy is expected to improve

safety of working conditions by increasing
working the number of workers in the
conditions solar sector and reducing the
number in the fossil fuel sector.

Diseases Yes No No The policy is not expected to

significantly affect these impact
Freedom of Yes No No categories, although reduced
expression energy costs may reduce poverty.

Access to safe Yes No No

drinking water

Poverty Yes No No

Gender Yes No No The policy is not expected

equality to significantly affect these
impact categories, although
gender equality is a high policy
priority, and some solar energy
policies will increase women’s
participation in the labour force
through new jobs, and women’s
entrepreneurship through new
business opportunities.

Mobility No No No This impact category is not

relevant to the assessment
or policy objectives and was
not expressed as a priority of
stakeholders.
 Part II: Defining the assessment 49

TABLE 5.2, continued

Example of reporting which impact categories are included in the assessment

for a solar PV incentive policy (hypothetical example only)

Included
in the Brief description
Impact assessment (rationale for determination
Dimension category Relevant? Significant? boundary? of relevance and significance)

Economic Jobs Yes Yes Yes The policy is expected to create

a significant number of new
jobs in the solar manufacturing,
installation and maintenance
sectors.

Income Yes Yes Yes The policy is expected to lead to

significant financial savings for
households, institutions and other
organizations through reduced
energy costs.

Wages No Yes No The policy is expected to increase

wages for workers in the solar
sector, but assessing wages is
not relevant to the objectives and
was not expressed as a priority of
stakeholders.

New business Yes Yes Yes The policy is expected to create

opportunities a significant number of new
business opportunities in the
solar manufacturing, installation
and maintenance sectors.

Energy Yes Yes Yes The policy is expected to lead to

independence significant improvement in energy
independence by reducing energy
imports.

Economic No No No The policy may affect these

activity impact categories, but the impact
is not expected to be significant.
Economic No No No They are also not relevant to the
productivity assessment or policy objectives
and were not expressed as a
Prices of No No No
priority of stakeholders.
goods and
services

Balance of No No No
payments

Note: This example is illustrative only. The impact categories that are relevant or significant for a solar PV incentive policy will depend
on the local context.
50 Sustainable Development Methodology

5.2 Identify indicators for each • United Nations Indicators of Sustainable

included impact category Development: Guidelines and Methodologies.21

An indicator is a metric that can be estimated to Indicators can be defined in a variety of ways for a
indicate the impact of a policy on a given impact given impact category. For example, to measure a
category, or can be monitored over time to enable policy’s impact on the number of jobs, indicators
tracking of changes towards targeted outcomes. could include the number of people employed, the
To assess impacts in later chapters, appropriate number of people unemployed, the employment
indicators need to be identified for each impact rate, the unemployment rate, the number of women
category that can be used to assess the impacts of and men employed, the number of short-term and
the policy. One or more indicators may be relevant long-term jobs, the number of full-time-equivalent
for each impact category. For example, if one of jobs, the number of jobs in various economic sectors,
the impact categories included in the assessment and the number of new jobs created. Additional
is “gender equality and empowerment of women”, indicators are needed to measure a policy’s impact
a user may select the indicators “average income on the quality of jobs, such as indicators related to
of women”, “number of women in the labour force” wages, benefits, job security and worker safety. Users
and “proportion of women in senior management can also decide whether to estimate the number of
positions” to assess the impact of the policy. direct jobs (e.g. the number of people installing solar
PV panels), indirect jobs (e.g. jobs involved in solar
It can be useful to identify indicators for qualitative panel manufacturing, distribution and marketing)
assessments (Chapters 6 and 7). Indicators for and/or induced jobs (e.g. jobs in other sectors, such
a qualitative assessment may be qualitative as food services supported by increased wages from
or quantitative. Indicators must be defined new solar PV installation jobs). As a conservative and
for quantitative assessments, because specific simplifying assumption, users may decide to only
indicators are estimated in the baseline and policy assess direct jobs.
scenarios (Chapters 8–10), and monitored over time
(Chapter 12). The choice of specific indicators, representing the
specific aspects of each impact category to be
For quantitative assessments, users should identify measured, should be based on the objectives of the
possible indicators at this stage to inform the assessment, in the context of what types of data are
qualitative assessment in Chapters 6 and 7. These available. When selecting appropriate indicators,
should be revisited after users have identified the users should consider the criteria outlined in
specific impacts of the policy in Chapter 6 and Table 5.3.
determined which are significant in Chapter 7.
The decision about which indicators to quantify is Users should consider defining indicators separately
described in Section 8.1. for various groups in society in addition to
aggregated statistics. For example, for the impact
category of jobs, users should consider defining
5.2.1 Selecting indicators indicators for the number of men and women
employed, in addition to the total number of people
Indicators should enable users to adequately assess employed, to show the impacts of a policy by gender.
whether a policy affects a given impact category, and As another example, since water scarcity and air
how. For guidance and examples of indicators that quality have locally specific impacts, users should
can be used, see: consider defining indicators for different regions
within a country to assess the local impacts of a
• United Nations SDG website17 policy on water scarcity or air quality. Indicators may
be disaggregated by gender, income groups, racial or
• United Nations SDG indicators website,18 ethnic groups, education levels, geographic regions,
including the global SDG indicators database19 urban versus rural, among others.
and list of indicators20
Table 5.4 provides examples of indicators that can be
disaggregated by gender.
17
https://sustainabledevelopment.un.org/sdgs
18
http://unstats.un.org/sdgs
19
http://unstats.un.org/sdgs/indicators/database 21
Available at: https://sustainabledevelopment.un.org/content/
20
http://unstats.un.org/sdgs/indicators/indicators-list documents/guidelines.pdf.
 Part II: Defining the assessment 51

TABLE 5.3

Criteria for selecting indicators

Criteria Description

Relevance Does the indicator measure what really matters, as opposed to what is easiest to measure? Users
should avoid measuring what is easy to measure instead of what is needed to meet the assessment
objectives.

Credibility How trustworthy or believable are the data to the intended audiences of the evaluation report?
Stakeholders and experts consulted may help identify credible sources of information. Technical
review of data can help improve credibility.

Validity Will the indicator reflect what the evaluator set out to measure? Validity refers to whether a
measurement actually measures what it is supposed to measure.

Reliability If data on the indicator are collected in the same way from the same source using the same decision
rules every time, will the same results be obtained? One way of improving reliability is ensuring that
monitoring occurs regularly.

Feasibility Users should avoid trying to measure too much. To limit the costs of data collection, users should
consider what indicators are already being monitored. Users should also consider whether the
indicator can be measured directly or whether (and how many) parameters are needed to calculate
the value of the indicator.

TABLE 5.4

Examples of indicators that can be disaggregated by gender

Impact category Indicators

Access to health-care • Proportion of women/men, girls/boys with health insurance or access to public health
services system

Hunger, nutrition and • Prevalence rate of undernourished girls/boys, women/men

food security

Illness and death • Life expectancy for women/men (years)

Access to safe drinking • Percentage of population (women/men) with access to safe drinking water
water

Access to adequate • Percentage of population (women/men) with access to sanitation facilities

sanitation

Access to clean, reliable • Percentage of population (women/men) with access to clean, reliable and affordable
and affordable energy energy

Access to land • Percentage of population (women/men) with access to land

Accessibility and quality • Proportion of girls/boys getting secondary school education

of education • Average years of schooling for girls/boys

Capacity, skills • Number of women/men, girls/boys who have received training

and knowledge
development
52 Sustainable Development Methodology

TABLE 5.4, continued

Examples of indicators that can be disaggregated by gender

Impact category Indicators

Climate change • Number of women/men, girls/boys who have received training

education, public
awareness, capacity-
building and research

Economic inequality • Average income for women/men

• Average wealth for women/men; difference in wealth between women and men
• Average wages for women/men; gender wage gap

Gender equality and • Average income for women/men

empowerment of • Gender wage gap
women
• Proportion of girls and women in schools
• Proportion of women in tertiary education
• Proportion of women in the labour force
• Proportion of women in senior management positions
• Proportion of women in senior government positions

Jobs • Number of women/men employed

• Number of women/men unemployed
• Employment rate for women/men
• Unemployment rate for women/men
• Number of jobs, including short-term jobs and long-term jobs, in different sectors for
women/men
• Number of new jobs created in different sectors for women/men

New business • Number of new companies headed by women/men

opportunities
 Part II: Defining the assessment 53

5.2.2 Examples of indicators

Table 5.5 provides examples of indicators for

selected impact categories in Table 5.1.

TABLE 5.5

Examples of indicators for selected impact categories

Impact category Indicators

Environmental impacts

Climate change mitigation • Net emissions of greenhouse gases (CO2, CH4, N2O, HFCs, PFCs, SF6 and NF3, and, if
(SDG 13) relevant, other gases identified by the IPCC) (t/year) and in carbon dioxide equivalent
(CO2e) using global warming potential
• Net emissions of short-lived climate pollutants (SLCPs): black carbon, organic carbon,
CO, NMVOCs, sulfates

Ozone depletion • Net emissions of ozone-depleting substances (such as CFC-11, CFC-113, halon 1211,
methyl chloroform) (t/year)
• Stratospheric ozone concentration (t/m3)

Air quality and health • Emissions of air pollutants such as particulate matter (PM2.5, PM10), ammonia, ground-
impacts of air pollution level ozone (resulting from VOCs and NOx), CO, SO2, NO2, fly ash, dust, lead, mercury and
(SDGs 3, 11, 12) other toxic pollutants (t/year)
• Concentration of air pollutants (mg/m3)
• Indoor and outdoor air quality (air quality index)
• Morbidity (DALYs, QALYs and ADALYs)
• Mortality (avoided premature deaths per year)

Visibility • Visual range (in units of distance)

• Deciview (dv)

Availability of fresh water • Water consumption (m3) or total amount of water removed from freshwater sources for
(SDG 6) human use
• Proportion of total water resources used (water scarcity)
• Water-use efficiency or intensity
• Stress-weighted water footprint (litres)

Water quality (SDGs 6, 14) • Net emissions of SO2, NOx, phosphorus, nitrogen, toxic pollutants (t/year)
• Acidity (pH)
• Accumulated exceedance
• Eutrophication from nutrient pollution (such as phosphorus and nitrogen compounds)
• Toxicity from emissions of toxic chemicals (e.g. metals, PAH)

Biodiversity of freshwater • Proportion of marine area protected

and coastal ecosystems • Proportion of fish stocks within safe biological limits
(SDGs 6, 14)
• Percentage of fish tonnage landed with maximum sustainable yield
• Damage on ecosystem (potential affected fraction of species)
• Marine trophic index
• Extinction rate
• Biodiversity intactness index
54 Sustainable Development Methodology

TABLE 5.5, continued

Examples of indicators for selected impact categories

Impact category Indicators

Environmental impacts, continued

Biodiversity of terrestrial • Species diversity (number of species or species richness)

ecosystems (SDG 15) • Change in threat status of species (abundance of selected key species, invasive alien
species or endangered species)
• Proportion of terrestrial area protected
• Damage to ecosystem (potential affected fraction of species)
• Extinction rate
• Biodiversity intactness index
• Quality of ecosystem services

Land-use change, • Annual change in degraded or desertified arable land (% or hectares)

including deforestation, • Area of forested land as a percentage of original or potential forest cover
forest degradation and
• Proportion of land area covered by forests
desertification (SDG 15)
• Area of forest under sustainable forest management
• Arable and permanent cropland area
• Area under organic farming

Soil quality • Net emissions of SO2, NH3 and NOx (t/year)

(SDG 2) • Soil organic matter
• Acidity (pH)
• Extent of soil erosion

Waste generation and • Solid waste generated (t/year)

disposal (SDG 12) • Wastewater generated
• Recycling rate (percentage of waste recycled)
• Proportion of materials reused
• Proportion of waste composted

Treatment of solid waste • Proportion of solid waste and wastewater safely treated
and wastewater (SDG 6)

Energy (SDG 7) • Energy consumption

• Energy efficiency
• Energy generated by source
• Renewable energy generation
• Renewable energy share of total final energy consumption
• Primary energy intensity of the economy (e.g. tonnes of oil equivalent/GDP)

Depletion of non- • Consumption of mineral resources

renewable resources • Consumption of fossil fuels
(SDG 12)
• Scarcity of resources

Toxic chemicals released • Emissions (t/year)

to air, water and soil

Genetic diversity and fair • Genetic diversity of seeds, plants and animals
use of genetic resources
(SDGs 2, 15)
 Part II: Defining the assessment 55

TABLE 5.5, continued

Examples of indicators for selected impact categories

Impact category Indicators

Environmental impacts, continued

Nuclear radiation • Human exposure efficiency relative to uranium-235

• Morbidity (DALYs)

Noise pollution • Noise level (decibels)

Social impacts

Accessibility and quality of • Proportion of people with health insurance or access to public health system
health care (SDG 3)

Hunger, nutrition and • Prevalence rate of undernourished people

food security (SDG 2) • Average share of food expenditures in total household expenditures
• Per capita total amount of net calories available in a given country
• Level of nutrition or malnutrition
• Agricultural crop diversity

Illness and death (SDG 3) • Life expectancy (years)

• Avoided premature deaths per year
• Morbidity (DALYs, QALYs and ADALYs)
• Maternal mortality
• Infant mortality
• Prevalence of diseases
• Proportion of population with diagnosed diseases or hospitalized from specific diseases
• Illnesses from hazardous chemicals, air pollution, water pollution and soil pollution
• Prevalence or reduction in respiratory illnesses
• Bioaccumulation of POPs and heavy metals

Access to safe drinking • Percentage of population with access to safe drinking water
water (SDG 6)

Access to adequate • Percentage of population with access to sanitation facilities

sanitation (SDG 6)

Access to clean, reliable • Percentage of population with access to clean, reliable and affordable energy
and affordable energy • Price of energy
(SDG 7)
• Emissions per unit of energy
• Number and length of service interruptions

Access to land (SDG 2) • Percentage of population with access to land

Standard of living • Gross national income per capita (adjusted according to PPP$)

Quality of life and well- • OECD Better Life Index

being (SDG 3) • Human Development Index
• Gross national happiness

Accessibility and quality of • Proportion of children getting primary and secondary school education
education (SDG 4) • Average years of schooling
56 Sustainable Development Methodology

TABLE 5.5, continued

Examples of indicators for selected impact categories

Impact category Indicators

Social impacts, continued

Capacity, skills and • Proportion of youth and adults with scientific, technological or other skills, by type of skill
knowledge development • Number of people who have received training
(SDGs 4, 12)

Climate change • Extent to which climate change education is mainstreamed in national education
education, public policies, curricula, teacher education and student assessment
awareness, capacity- • Proportion of population aware of climate change
building and research
• Number of people who have received training

Quality of institutions • Effectiveness of institutions

(SDG 10) • Credibility of institutions
• Accountability of institutions
• Legitimacy of institutions

Poverty (SDG 1) • Poverty rate (proportion of population living below national poverty line)
• Proportion of people living on less than $1.25 (SDGs), $1.90 (World Bank) or other
amount per day
• Number of people living in poverty
• Multidimensional poverty index
(see http://hdr.undp.org/sites/default/files/hdr2015_technical_notes.pdf)

Economic inequality • Income equality/inequality, average income for different groups, share of national
(SDGs 8, 10) income by income quintile
• Wealth equality/inequality, average wealth for different groups, share of national wealth
by wealth quintile
• Wage equality/inequality, average wages for different groups

Gender equality and • Average income for women and men

empowerment of women • Gender wage gap
(SDG 5)
• Proportion or number of girls and women in schools
• Proportion or number of women in tertiary education
• Proportion or number of women in the labour force
• Proportion or number of women in senior management positions
• Proportion or number of women in senior government positions
• Women’s decision-making power within family/community
• Women’s ability to spend income earned

Racial equality • Average income by racial/ethnic group

• Proportion of people in schools by racial/ethnic group
• Proportion of people in the labour force by racial/ethnic group
• Proportion of people in senior management positions by racial/ethnic group

Indigenous rights • Extent of recognition of ancestral land titles

• Extent of free, prior and informed consent
• Extent of protection of indigenous traditional knowledge
• Extent of empowerment of indigenous communities
 Part II: Defining the assessment 57

TABLE 5.5, continued

Examples of indicators for selected impact categories

Impact category Indicators

Social impacts, continued

Mobility (SDG 11) • Number of people or proportion of population with convenient access to employment,
schools, health care or recreation, by gender, age and persons with disabilities

Traffic congestion • Time lost during transportation

(SDG 11) • Economic cost of time lost

Road safety (SDGs 3, 11) • Number of deaths and injuries from road traffic accidents per year

Resilience to dangerous • Creation and maintenance of climate-resilient infrastructure

climate change and • Reduction of natural disaster risks
extreme weather events
(SDG 13)

Economic impacts

Economic activity (SDG 8) • GDP

• Gross national income
• Local or state/provincial GDP
• Annual growth rate of real GDP per capita

Economic productivity • Agricultural productivity (harvested crop yields per hectare)

(SDGs 2, 8)

Jobs (SDG 8) • Number of people employed

• Number of people unemployed
• Employment rate
• Unemployment rate
• Number of jobs, including short-term jobs and long-term jobs, in different sectors
• Number of new jobs created in different sectors

Wages (SDG 8) • Average hourly wage (nationally or in different economic sectors)

• Average hourly wage for different groups (by gender, income, etc.)

Worker productivity • Labour productivity per hour or per unit of labour

• Total employment or number of hours worked per GDP

New business • Number of new companies

opportunities (SDG 8) • Revenue and profit
• Amount of new investment
• Number of active long-term partnerships

Growth of new • Amount of investment in clean technology sector

sustainable industries • Revenue and profit from clean technology sector
(SDGs 7, 17)
• Number of projects

Competitiveness of • Market share

domestic industry in • Quantity/value of exports
global markets
• Balance of trade
58 Sustainable Development Methodology

TABLE 5.5, continued

Examples of indicators for selected impact categories

Impact category Indicators

Economic impacts, continued

Economic development • Revenue from tourism

from tourism and • Tourism GDP as a proportion of total GDP
ecotourism (SDG 8)
• Number of jobs in tourism industries as a proportion of total jobs, and growth rate of
jobs (by women/men)

Income (SDG 10) • Income per capita

• Median household income
• Annual growth in household income

Prices of goods and • Energy prices

services

Costs and cost savings • Fuel costs or cost savings

• Health-care costs or cost savings
• Economic costs of human health losses from air pollution based on social welfare
indicator (ADALYs monetized in terms of social welfare valuation based on willingness to
pay VSL estimates) or national accounts indicator (ADALYs monetized based on foregone
output estimates based on productivity/wage approaches)

Inflation • Inflation rate

Balance of trade • Total imports

• Total exports
• Net imports

Government budget • Annual revenue

surplus/deficit • Annual expenditures
• Annual surplus or deficit

Energy independence • Net imports of fossil fuels (coal, oil, natural gas)

Abbreviations: ADALY, averted disability-adjusted life year; CFC, chlorofluorocarbon; CH4, methane; CO, carbon monoxide;
CO2, carbon dioxide; DALY, disability-adjusted life year; GDP, gross domestic product; HFC, hydrofluorocarbon; NF3, nitrogen
trifluoride; NH3, ammonia; NMVOC, non-methane volatile organic compound; N2O, nitrous oxide; NO2, nitrogen dioxide;
NOx, nitrogen oxides; OECD, Organisation for Economic Co-operation and Development; PAH, polycyclic aromatic hydrocarbon;
PFC, perfluorocarbon; POP, persistent organic pollutant; PPP, purchasing power parity; QALY, quality-adjusted life year; SF6, sulfur
hexafluoride; SO2, sulfur dioxide; VOC, volatile organic compound; VSL, value of statistical life
PART III
Qualitative approach to impact
­assessment
6 Identifying specific impacts within each
impact category

After choosing which impact categories to assess in 6.1 Identify specific impacts of the
Chapter 5, the next step is to identify the specific impacts policy within each impact category
within each selected impact category. This chapter
explains how to identify all potential impacts of a policy A comprehensive understanding of impacts is crucial
within each sustainable development impact category to the completeness and accuracy of the assessment.
that has been included in the assessment boundary. For each impact category included in the assessment
boundary in Chapter 5, it is a key recommendation
This step is relevant for all users – both those following to identify all potential sustainable development
qualitative and those following quantitative approaches impacts of the policy within each impact category
– and for either ex-ante or ex-post assessment. For all included in the assessment, using a causal chain and
users, the set of impacts identified in this chapter will table format, if relevant and feasible, in consultation
be included in the qualitative assessment boundary and with stakeholders.
qualitatively assessed in Chapter 7. For users following
a quantitative approach, it is not necessary to estimate If significant sustainable development impacts are
all the impacts identified in this chapter. Instead, the identified during this step that were not considered
qualitative assessment step in Chapter 7 will be used to in Chapter 5, users should consider revising the list of
determine which impacts are significant, and therefore impact categories included in the assessment.
recommended to be included in the quantitative
assessment boundary and estimated (in Chapter 8). It
is important to comprehensively consider all potential 6.1.1 Types of specific impacts
impacts in this chapter before setting the quantitative
assessment boundary. To identify sustainable development impacts, it can
be useful to first identify the intermediate impacts
Checklist of key recommendations resulting from the policy that lead to sustainable
development impacts. “Intermediate impacts” are
• Identify all potential sustainable development changes in behaviour, technology, processes or
impacts of the policy within each impact practices that result from the policy and lead to
category included in the assessment, using a sustainable development impacts. “Sustainable
causal chain and table format, if relevant and development impacts” are changes in specific
feasible, in consultation with stakeholders sustainable development impact categories, such
• Separately identify and categorize in- and as changes in air quality, jobs or health, among
out-of-jurisdiction sustainable development others outlined in Chapter 5. Figure 6.2 illustrates
impacts, if relevant and feasible the relationship between intermediate impacts and
sustainable development impacts.

FIGURE 6.1

Overview of steps in the chapter

Identify specific impacts of the policy within

Describe and report specific impacts
each impact category
(Section 6.2)
(Section 6.1)
 Part III: Qualitative approach to impact assessment 61

Sustainable development impacts are the impacts single impact category of jobs: an increase in
of interest (such as increased jobs in the solar jobs in the solar installation, operations and
manufacturing sector), whereas intermediate maintenance sectors; an increase in jobs in the solar
impacts lead to an impact of interest (such as manufacturing sector; an increase in jobs in the solar
increased demand for solar PV systems, which and grid technology sectors, including mining of rare
leads to increased solar PV manufacturing). Both earth minerals for solar cells; a decrease in jobs in
intermediate and sustainable development impacts the fossil fuel power plant design, operations and
can be short term or long term. maintenance sectors; and a decrease in jobs in fossil
fuel sectors.
An intermediate impact in one context may be
a sustainable development impact in another To ensure a complete assessment, users should
context, depending on the policy objectives and consider a wide range of potential impacts, as
circumstances. For example, cost savings may be a outlined in Table 6.1. It is important to identify
sustainable development impact in one context and, not only positive and intended impacts, but also
in another context, an intermediate impact towards potential negative and unintended impacts, to
using the savings to achieve improved nutrition, comprehensively assess the total net impact of
health care, education or quality of life. the policy on the impact categories included in
the assessment. In Chapter 7, each impact will be
Each impact category included in the assessment qualitatively assessed to determine whether it is
may have multiple distinct impacts. For example, significant. Insignificant impacts will be excluded
a solar PV incentive policy may have five distinct from the quantitative assessment boundary (for
sustainable development impacts within a users following a quantitative approach).

FIGURE 6.2

Intermediate impacts and sustainable development impacts

SUSTAINABLE
POLICY INTERMEDIATE IMPACTS
DEVELOPMENT IMPACTS

TABLE 6.1

Types of impacts, definitions and examples

Type of impact Definition Examples for a solar PV incentive policy

Positive and Impacts that are perceived as favourable or Positive: Reduced air pollution from
negative unfavourable from the perspectives of different distributed fossil fuel generation
stakeholder groups Negative: Increased air pollution from solar
production, transportation and installation

Intended and Impacts that are intentional or unintentional, Intended: Reduced air pollution from
unintended based on the original objectives of the policy, distributed fossil fuel generation
and from the perspective of policymakers and Unintended: Increased air pollution from
stakeholders (In some contexts, intentional solar production, transportation and
impacts are called primary impacts and installation
unintended impacts are called secondary
impacts.)
62 Sustainable Development Methodology

TABLE 6.1, continued

Types of impacts, definitions and examples

Type of impact Definition Examples for a solar PV incentive policy

Short term and Impacts that are nearer or more distant in Short term: Increased renewable energy
long term time, based on the amount of time between generation from more solar generation
implementation of the policy and the impact Long term: Increased energy independence
from reduced imports of fossil fuels

In-jurisdiction Impacts that occur inside the geopolitical In-jurisdiction: Increased domestic jobs for
and out-of- boundary over which the implementing entity solar installation, operations and maintenance
jurisdiction has authority, such as a city boundary or Out-of-jurisdiction: Increased jobs in other
national boundary, and impacts that occur countries for solar manufacturing, since solar
outside the geopolitical boundary PV is imported

Technology Changes in technology such as design or Replacement of diesel generators with solar
deployment of new technologies PV technology

Business and Changes in business practices or behaviour Business: Increased business opportunities
consumer (such as manufacturing decisions), and for solar manufacturing, mining,
consumer practices or behaviour (such as transportation, solar power plants and grid-
purchasing decisions) associated technologies
Consumer: increased disposable household
income due to a reduction in energy costs.

Infrastructure Changes in existing infrastructure or Reduced GHG emissions associated with

development of new infrastructure decreased manufacturing of new fossil fuel
generation plants

Market Changes in supply and demand, prices, market Increased business opportunities for solar
structure or market share installation, operations and maintenance

Life cycle Changes in upstream and downstream Increased air pollution from solar PV
activities, such as extraction and production of production, transportation and installation
energy and materials, or impacts in sectors not
targeted by the policy

Macroeconomic Changes in macroeconomic conditions, such Increased household and business income
as GDP, income or employment, or structural and spending due to reduction in energy
changes in economic sectors costs

Trade Changes in imports and exports Reduced imports of fossil fuels

Institutional Changes in institutional arrangements Establishment of a new government unit to

implement the solar PV incentive policy

Distributional Changes in how income, resources or costs are Increased income for households, institutions
distributed among a population, or changes and other organizations that install solar PV
among different demographic groups, such as systems
gender or income groups

Source: Adapted from WRI (2014).

 Part III: Qualitative approach to impact assessment 63

The types of impacts in Table 6.1 are intended to Each specific impact should be characterized relative
guide the development of a comprehensive list to a baseline scenario – that is, the conditions
of potential impacts. The types of impacts are not most likely to occur in the absence of the policy.
mutually exclusive, so each impact will fit into multiple For example, in a country where coal production is
types. For example, a single impact may be positive, increasing significantly over time, jobs in the coal-
intended, in-jurisdiction and long term. Table 6.1 mining sector may continue to increase even with
provides users with different lenses to view impacts in a new solar PV incentive policy. However, jobs in
different ways, to help identify all potential impacts of the coal-mining sector would have increased by
the policy. However, the list is neither prescriptive nor a greater amount if the new solar policy did not
exhaustive, and not all types of impacts listed may be exist, since the policy reduces demand for coal
relevant to the policy being assessed. relative to the baseline scenario. Therefore, the user
should identify the impact as a decrease in jobs in
In-jurisdiction and out-of-jurisdiction impacts the coal-mining sector resulting from the solar PV
It is a key recommendation to separately identify and incentive policy, even though there is no decrease
categorize in- and out-of-jurisdiction sustainable in absolute terms. In Chapters 6 and 7, users should
development impacts, if relevant and feasible. Users identify and characterize impacts relative to baseline
should define the jurisdictional boundary based on scenarios in conceptual terms, even if baseline
what is most relevant, and be transparent about scenarios are not explicitly defined. Chapter 8
which jurisdictional boundary is used. provides detailed guidance on estimating baseline
values in a quantitative assessment and may also be
Separately tracking in- and out-of-jurisdiction useful when identifying impacts relative to baseline
impacts can help link the policy or action to the scenarios.
implementing jurisdiction’s sustainable development
goals by separating the impacts that affect the Causal chain
jurisdiction’s goals from impacts that occur outside A causal chain is a conceptual diagram tracing
the jurisdiction. Separate tracking can also address the process by which a policy leads to various
potential double counting of out-of-jurisdiction sustainable development impacts through a series
impacts between jurisdictions. of interlinked logical and sequential stages of
cause-and-effect relationships. Developing a causal
Out-of-jurisdiction impacts may be especially chain is a useful tool for identifying, organizing
relevant for subnational policies that have impacts in and communicating all potential sustainable
other subnational regions within the same country. development impacts of the policy. It helps users and
Transnational impacts in neighbouring countries may stakeholders understand the logic and underlying
also be relevant. Where collecting data from other assumptions of impacts by showing how the policy
jurisdictions is difficult, users may need to estimate leads to changes through a series of intermediate
impacts rather than using the more accurate data- impacts. To identify a comprehensive list of impacts,
collection methods that can be used within the users should develop a causal chain that includes all
implementing jurisdiction. potential impacts of the policy within each impact
category included in the assessment, to the extent
If a single impact is both in-jurisdiction and feasible.
out-of-jurisdiction and separate tracking is not
feasible, users can apportion the impact between To develop the causal chain, users should first
in-jurisdiction and out-of- jurisdiction based on identify the proximate (first-stage) intermediate
assumptions. impacts of the policy. It may be useful to first
consider the inputs, resources and activities
involved in implementing the policy to help identify
6.1.2 Methods for identifying and organizing the proximate impacts, or changes in behaviour,
specific impacts technology, processes or practices. Each first-stage
impact represents a distinct “branch” of the causal
A variety of methods may be used to identify specific chain. Each branch of the causal chain may lead to
impacts resulting from a policy, including developing one or more intermediate impacts or sustainable
a causal chain and using an impact matrix table. For development impacts. Users should extend each
either method, stakeholder consultation, literature branch of the causal chain through a series of
review and expert judgment can be used to identify cause-and-effect relationships – that is, a series of
impacts. The methods are not mutually exclusive intermediate effects – until the causal chain leads to
and should be used in combination to identify all all potential sustainable development impacts in the
potential impacts. selected impact categories, to the extent feasible.
64 Sustainable Development Methodology

Figure 6.3 provides an example of a causal chain for chains for each impact category. Where the number
a solar PV incentive policy that includes intermediate of impact categories is relatively small and where
impacts and sustainable development impacts for impact categories are interrelated, users may find
one impact category: jobs. Users should identify all it useful to include all sustainable development
intermediate impacts that may lead to sustainable impact categories in a single, integrated causal
development impacts, and as many sustainable chain. A single causal chain can help stakeholders
development impacts as possible, considering the understand all impact categories in a single diagram
types of impacts in Table 6.1. and the relationships between impact categories. On
the other hand, if the impact categories included in
It is possible that a sustainable development impact the assessment are less closely related and do not
in one category may lead to another sustainable have many intermediate impacts in common, or if
development impact in another category. For developing an integrated causal chain would be too
example, an increase in household income (a complex, users can develop separate causal chains
sustainable development impact relating to income) for each selected impact category.
that results from a solar PV incentive policy may
lead to increased demand for goods and services, Figure 6.4 provides an example of a causal chain
which may lead to increased economic activity that includes multiple impact categories. It can be
(a sustainable development impact relating difficult to include all impact categories and specific
to economic activity). Box 5.2 provides more impacts within a single causal chain, depending on
information on interlinkages between related the number of impact categories and specific impacts
sustainable development impact categories. identified. Figure 6.4 includes all impact categories
included in the assessment, but does not include
In different situations, it may be more appropriate to all specific impacts within each impact category.
develop either (1) a single causal chain that contains Figure 6.5 separately illustrates social and economic
all sustainable development impact categories impacts, rather than combining them in a single
included in the assessment, or (2) separate causal diagram.

FIGURE 6.3

Example of a causal chain for the jobs impact category

FIRST STAGE SECOND STAGE THIRD STAGE FOURTH STAGE

Decreased jobs in
Decreased demand fossil fuel power
Increased electricity for centralized grid- plant design,
generation from connected electricity operations and
rooftop solar systems (from coal and maintenance
natural gas) sectors

Decreased
Increased jobs in
Increased installation Decreased demand jobs in fossil
solar installation,
Solar PV incentive of solar PV systems for distributed fuel extraction,
operations and
policy by households due generation (from transportation
maintenance
to lower cost diesel generators) and import/export
sectors
sectors

Increased jobs
in solar and grid
technology sectors,
and mining of rare
earth minerals for
Policy solar cells

Intermediate effect
Increased
jobs in solar
Jobs impact manufacturing
and transportation
sectors
 Part III: Qualitative approach to impact assessment 65

FIGURE 6.4

Example of a causal chain that includes all impact categories included in the assessment

Decreased Reduced GHG

demand emissions
for grid- from fossil fuel
connected generation and
electricity extraction
(from coal
and natural
gas) Reduced Reduced
fossil fuel air pollution
consumption from reduced
Decreased fossil fuel
demand for generation
distributed
generation Reduced Increased
(from diesel imports of energy
generators) fossil fuels independence

Increased Decreased
Increased
GHG emissions business
access to
from solar PV opportunities Decrease in
electricity
production for fossil fuel training for
due to lower
sectors skilled workers
cost of solar
Increased power for self- in fossil fuel
installation Increased Increased Decreased sectors
consumption
Solar PV of solar PV renewable air pollution jobs in fossil
incentive systems by energy from solar PV fuel extraction,
policy households generation production transportation, Reduced
Increased income from
due to lower from solar PV import/
household reduced fossil
cost export, and
disposable Increased fuel jobs
power plant
income due waste
operations and
to reduced generation
maintenance
electricity costs from solar Increased
panel mining safety and
Policy and production Decreased working
waste conditions due
Increased generation to fewer jobs
Intermediate effect production Increased and disposal in fossil fuel
Increased of solar PV jobs in solar from fossil fuel sector
business manufacturing mining and
Jobs opportunities generation
and silica
for solar- mining sectors
related
Income Increase in
sectors Increased training for
mining of Increase in skilled workers
Climate change mitigation silica training for in solar
skilled workers manufacturing
in solar
Air quality installation
Increased
income from
Energy
Increased increased jobs
income from
Waste increased jobs
Reduced
safety/working
New business opportunities conditions due
Increased Improved to more jobs
jobs in solar safety/working in silica mining
Access to clean, affordable, reliable energy installation, conditions and solar
operations due to more manufacturing
and jobs in solar
Quality and safety of working conditions maintenance installation
sectors sector

Capacity, skills and knowledge development

Energy independence

Note: This example includes all impact categories included in the assessment but does not include all identified specific impacts within
each impact category.
66 Sustainable Development Methodology

FIGURE 6.5

Example of causal chains that separately illustrate social and economic impacts

Increased Decreased Decreased

Increased
renewable electricity jobs related
Intermediate impacts health
electricity consumption to electricity
conditions for
consumption from national generation of
workers
Social impacts from PV panels grid national grid

Increased
Increased Risks of Increased
manufacturing
installation of PV decreased health safety
& raw material
panels & LEDs conditions for conditions for
extraction for PV
workers workers
panels & LEDs

Increased
climate change Increased Risks of Decreased
Public awareness of transport of PV decreased safety training of local
buildings civil servants panels & LEDs conditions for skilled workers
energy workers
efficiency
project Increased Increased waste
(pilot test) acceptance of processing & Increased
renewables disposal for PV training of local
from investors panels & LEDs skilled workers

Increased Increased local

Increased local
acceptance of R&D related to
jobs from PV
renewables by energy
panels installation
general public

Increased
lighting quality
for task
visibility in
offices

Increased Decreased
Decreased jobs
renewable electricity
from current
electricity consumption
electricity
consumption from national
generation
from PV panels grid

Increased
Increased
rebound
manufacturing Increased jobs Increased
investments
& raw material from PV panels & electricity cost
that impact
extraction for PV LEDs production savings
sustainable
panels & LEDs
development
Public
buildings
Increased Increased Increased jobs
energy
installation of PV transport of PV from PV panels &
efficiency
panels & LEDs panels & LEDs LEDs transport
project
(pilot test)
Increased jobs
Increased waste
from PV panels
processing &
& LEDs waste
disposal for PV
processing
panels & LEDs

Increased
local jobs from
Increased
PV panels
opportunities for
Intermediate impacts installation
energy-related
local businesses
Economic impacts Increased
income for local
economy
Source: Cuesta Claros (2018).
 Part III: Qualitative approach to impact assessment 67

If useful, the causal chain can be colour-coded 6.1.3 Literature review, stakeholder
or include symbols to designate different impact consultations and expert judgment
categories or types of impacts, such as positive
versus negative impacts or in-jurisdiction versus out- Users should review literature and conduct
of-jurisdiction impacts. stakeholder consultations when identifying impacts
and developing a causal chain or impact matrix table.
The causal chain should be as comprehensive Users can also use expert judgment to supplement
as possible, rather than limited by geographic or these efforts.
temporal boundaries. To make the process more
practical, users should only include those branches To the extent feasible, users should review prior
of the causal chain that are reasonably expected assessments or case studies of similar policies and
to lead to sustainable development impacts in impact categories. Additional literature that may
categories selected for assessment. If the causal be useful includes regulations, development plans,
chain becomes too complex, users can summarize regulatory impact analyses, environmental impact
the sustainable development impacts for each assessments, risk assessments and economic
branch without mapping each intermediate impact studies. It may also be useful to refer to guidance
for each stage separately. or methods that are sector-specific and/or impact-
category-specific. The ICAT website provides
Impact matrix table references to methods and models for assessing
Users may also find it helpful to develop an impact specific impacts, which can help users identify
matrix table to identify specific impacts. To do so, impacts.22
users should select a set of impact types to put in the
column headers and a different set of impact types Users should also consult relevant experts and
in the row headers. Users then identify impacts for stakeholders when identifying impacts and
each combination of impact types. Table 6.2 provides constructing the causal chain. Different stakeholder
an example. Users can develop multiple impact groups approach a policy from different perspectives.
matrix tables for the policy to ensure that all impacts By conducting stakeholder consultations to identify
are identified. Note that the purpose of the table is to impacts, users can enhance the completeness of
help identify all potential impacts; whether a specific the impacts identified, identify and address possible
impact is classified as one type of impact or another unintended or negative impacts early on, and
is less important than developing a comprehensive increase acceptance of the final assessment results.
list of potential impacts. Stakeholder consultation may include interviews,
surveys or focus groups. Chapter 8 of the ICAT
Stakeholder Participation Guide provides information
on how to consult stakeholders.

TABLE 6.2

Example of an impact matrix table for an illustrative solar PV incentive policy

for the jobs impact category

Type of impact Short term Long term

Intended Increased jobs in domestic solar PV Increased jobs in domestic solar PV

installation, operations and maintenance manufacturing sector
sectors

Unintended Reduced jobs in domestic fossil fuel sector

Note: Increases in jobs are in green, and decreases in jobs are in red.

22
https://climateactiontransparency.org/icat-toolbox/sustainable-
development
68 Sustainable Development Methodology

6.2 Describe and report specific To provide clarity for each identified impact, users
impacts should describe the direction of change (increase
or decrease), and the underlying logic and causal
Communicating all identified impacts helps relationship of how the impact is expected to occur.
stakeholders understand the various impacts of the For example, impacts on jobs resulting from a solar
policy, and helps users determine the most relevant PV incentive policy may include an “increase in jobs
impacts to assess in a transparent and consistent in solar manufacturing due to increased demand”,
manner. This is important to enable decision makers an “increase in jobs in solar PV installation due to
to take actions to address any negative impacts and increased demand” and a “decrease in jobs in the coal-
enhance positive impacts. mining sector due to decreased demand”. The level of
detail will depend on the user’s objectives and context.
Users should report all identified sustainable
development impacts using a causal chain and a When reporting impacts using a table format, users
table format, if relevant and feasible. Reporting should report all identified sustainable development
impacts using a causal chain helps users and decision impacts but, to keep the report simple for readers, it is
makers understand in visual terms how the policy not necessary to include intermediate impacts. Users
leads to changes across sustainable development should specify the impact category for each impact
impact categories. This can be useful for enhancing and whether it is in-jurisdiction, out-of-jurisdiction or
policy design, improving understanding of policy mixed. If it would be helpful, users can report the type
effectiveness and communicating the impacts of the of impact, such as intended or unintended, short term
policy to stakeholders. Reporting the impacts using or long term, or positive or negative, and the methods
a table format, such as the reporting template, helps or sources used to identify each impact. Table 6.3
users undertake the steps in the following chapters provides a reporting template that can be used to
by using a single template. report the identified impacts, using an illustrative
example of a solar PV incentive policy.

TABLE 6.3

Example of reporting impacts using reporting template for a solar PV incentive policy

Impact
categories Methods/
included sources
in the used to
assessment In- or Type of identify
(from Specific impacts identified (within each impact out-of- impacts impacts
Chapter 5) category) jurisdiction (optional) (optional)

Climate Reduced GHG emissions from grid-connected fossil In

change fuel–based power plants
mitigation
Reduced GHG emissions from distributed fossil fuel In
generation

Reduced GHG emissions associated with In

manufacturing of new fossil fuel generation plants

Reduced GHG emissions from fossil fuel extraction Both

and transportation

Increased GHG emissions from solar PV production Both

Increased GHG emissions from solar PV transportation In

and installation

Increased GHG emissions from increased production In

of goods and services due to increased income
 Part III: Qualitative approach to impact assessment 69

TABLE 6.3, continued

Example of reporting impacts using reporting template for a solar PV incentive policy

Impact
categories Methods/
included sources
in the used to
assessment In- or Type of identify
(from Specific impacts identified (within each impact out-of- impacts impacts
Chapter 5) category) jurisdiction (optional) (optional)

Air quality/ Reduced air pollution from grid-connected fossil fuel– In

health based power plants
impacts of
air pollution Reduced air pollution from distributed fossil fuel In
generation

Reduced indoor air pollution from traditional use of In

biomass

Reduced air pollution from manufacturing of new fossil In

fuel generation plants

Reduced air pollution from fossil fuel extraction and Both

transportation

Increased air pollution from solar PV production Both

Increased air pollution from solar PV transportation Both

and installation

Increased air pollution from increased production of In

goods and services due to increased income

Waste Decreased waste generation and disposal from In

generation reduced fossil fuel generation (e.g. coal ash)
and
disposal Decreased waste generation and disposal from Both
reduced fossil fuel production and transportation

Increased waste generation and disposal from Both

increased solar mining and panel production (e.g.
silicon tetrachloride waste)

Increased waste generation and disposal from In

discarded solar panels (e.g. cadmium and tellurium)

Renewable Increased renewable energy generation from In

energy increased solar generation
generation

Access Increased access to clean, affordable and reliable In

to clean, electricity
affordable
and reliable Decreased access to electricity due to fewer new coal In
energy power plants

Capacity, Increase in training for skilled workers in solar-relevant Both

skills and sectors
knowledge
development Decrease in training for skilled workers in fossil fuel Both
sectors
70 Sustainable Development Methodology

TABLE 6.3, continued

Example of reporting impacts using reporting template for a solar PV incentive policy

Impact
categories Methods/
included sources
in the used to
assessment In- or Type of identify
(from Specific impacts identified (within each impact out-of- impacts impacts
Chapter 5) category) jurisdiction (optional) (optional)

Quality and Increased safety and working conditions due to more In

safety of jobs in the solar installation sector, where workers
working have better working conditions
conditions
Increased safety and working conditions due to fewer Both
jobs in the coal sector, where workers have worse
working conditions

Decreased safety and working conditions due to more Both

jobs in silica mining and solar cell manufacturing,
where workers have worse working condition (e.g. the
lung disease silicosis, exposure to hydrofluoric acid
and cadmium)

Jobs Increased jobs in the solar installation, operations and In

maintenance sectors

Increased jobs in the solar panel manufacturing sector Both

Increased jobs in the solar and grid technology sectors, Both

and mining of rare earth minerals for solar cells

Decreased jobs in the fossil fuel power operations and In

maintenance sectors

Decreased jobs in fossil fuel sectors Both

Decreased job in fossil fuel generation technology Both

sectors (e.g. supercritical and ultra-supercritical
generation)

Income Increased income for households, institutions and In

other organizations due to reduction in energy costs

New business Increased business opportunities for solar Both

opportunities manufacturing, mining, transportation, solar power
plants and grid-associated technologies

Decreased business opportunities for fossil fuel Both

extraction and transportation, fossil fuel power plants,
and fossil fuel–generated associated technologies

Energy Increased energy independence from reduced imports In

independence of fossil fuels (e.g. oil and gas)

Decreased energy independence from foreign control In

over scarce resources needed to manufacture solar
panels
7 Qualitatively assessing impacts

This chapter provides guidance on assessing sustainable Checklist of key recommendations

development impacts qualitatively. This step is relevant
for users who are following either a qualitative or a • Include all impact categories included in
quantitative approach, and for either ex-ante or ex-post Chapter 5 and all specific impacts identified
assessment. The chapter explains how to qualitatively in Chapter 6 in the qualitative assessment
assess each specific impact identified in Chapter 6 and boundary
summarize the qualitative assessment results for each • Define the assessment period
impact category. • Characterize each identified impact identified
in Chapter 6 based on the likelihood that
For users following a quantitative approach, this each impact will occur, the magnitude of each
qualitative step is used to prioritize which specific impact and the nature of the change (positive
impacts to quantify in later chapters. The quantitative or negative)
assessment boundary (defined in Chapter 8) should • Based on the assessment of likelihood and
include all impacts determined to be significant based magnitude, determine which identified
on the qualitative assessment in this chapter, where impacts are significant, in consultation with
feasible. stakeholders
• Summarize the qualitative assessment results
for each impact category, taking into account
all significant impacts
• Separately assess the impacts of the policy on
different groups in society, where relevant

7.1 Introduction to qualitative

assessment

Qualitative assessment is an impact assessment

approach that involves describing the impacts of a
policy on selected impact categories in qualitative
terms. This is in contrast to quantitative assessment,

FIGURE 7.1

Overview of steps in the chapter

Characterize each Summarize

Define the qualitative
Introduction specific impact in the qualitative
assessment
to qualitative terms of likelihood, assessment results
boundary and
assessment magnitude and for each impact
period
(Section 7.1) nature of the change category
(Section 7.2)
(Section 7.3) (Section 7.4)
72 Sustainable Development Methodology

which involves estimating the impacts of a policy on all impact categories included in Chapter 5 and
selected impact categories in quantitative terms. all specific impacts identified in Chapter 6 in the
qualitative assessment boundary.
Qualitative assessment is simpler and requires fewer
resources than quantitative assessment (outlined Both short-term and long-term impacts may result
in later chapters). In some cases, the qualitative from a policy, as identified in Chapter 6. It is a key
approach to impact assessment may be sufficient recommendation to define the assessment period.
to meet the stated objectives of the assessment. The assessment period is the time period over which
However, the qualitative approach does not enable impacts resulting from the policy are assessed. The
an accurate or quantified estimate of the impacts of assessment period can be shorter or longer than the
a policy, which limits its ability to meet a wider set of policy implementation period (i.e. the period during
objectives relating to understanding policy impact which the policy is in effect).
with greater certainty.
For an ex-ante assessment, users should consider the
A qualitative assessment can use both qualitative assessment objectives and stakeholders’ needs when
and quantitative data. Qualitative data can be used to determining the assessment period. For example, a
describe concepts that are harder to measure, such five-year assessment period may be appropriate if
as quality, behaviour or experiences. Quantitative the objective is to inform policymakers on sustainable
data can be used to measure or estimate quantities development progress by the end of a five-year
such as cost, time, area and energy. Whereas planning cycle. If the objective is to understand the
quantitative data can show how a policy is progressing expected contribution of the policy towards achieving
and whether it has led to a given impact, qualitative a country’s NDC, it may be most appropriate to align
methods (e.g. stakeholder interviews, focus groups, the assessment period with the NDC implementation
case studies) can show a more nuanced story of period (e.g. ending in 2030). Similarly, to align the
change, such as how or why a change happened results with the achievement of SDGs under the
for specific stakeholders, who has benefited and 2030 Agenda for Sustainable Development, users
why, and experiences or impacts for different may define an assessment period ending in 2030.
stakeholder groups. This qualitative information can To align with longer-term trends and planning,
help policymakers improve the policy over time. It users should select an end date such as 2040 or
can provide additional insights into a policy’s specific 2050. If the objective is to have a comprehensive
local context and impacts, from experiences and understanding of all impacts resulting from the policy,
perspectives of affected stakeholders. the assessment period should be based on when the
full range of impacts are expected to occur.
In certain cases, qualitative assessments can be
more subjective and uncertain than quantitative For an ex-post assessment, the assessment period
assessments. They can therefore lead to inaccurate can be the period between the date the policy is
and misleading results if they are not combined with implemented and the date of the assessment, or
a quantitative assessment. Depending on the level of a shorter period between these two dates. The
sampling of different stakeholder groups, qualitative assessment period for a combined ex-ante and
assessments can also be limited in coverage and ex-post assessment should consist of both an ex-
therefore not representative of broader conditions ante assessment period and an ex-post assessment
or impacts, which can produce less reliable results period.
and less ability to generalize impacts. Therefore, it
can be helpful to use a combination of qualitative In addition, users can separately estimate and
and quantitative data and approaches. For more report impacts over any other time periods that
information on qualitative methods, see Appendix C. are relevant. For example, if the assessment period
is 2020–2040, a user may separately estimate and
report impacts over the periods 2020–2030, 2030–
2040 and 2020–2040.
7.2 Define the qualitative
assessment boundary and period If an appropriate assessment period cannot easily be
determined, users can use short-term, medium-term
The qualitative assessment boundary defines the or long-term classifications to define the assessment
scope of the qualitative assessment in terms of period. Table 7.1 provides rules of thumb for
the range of dimensions, impact categories and assessment period lengths. Users can also define the
specific impacts that are included in the qualitative time periods differently; in this case, users should
assessment. It is a key recommendation to include report the time periods used.
 Part III: Qualitative approach to impact assessment 73

TABLE 7.1

Rules of thumb for ex-ante assessment periods

Assessment period Approximate assessment period

Short term <5 years

Medium term ≥5 years and <15 years

Long term ≥15 years

Users who are assessing the GHG impacts and/or • Step 2. Assess the expected magnitude of
transformational impacts of the policy, following each sustainable development impact.
other ICAT methodologies, should align the
assessment periods between the assessments to • Step 3. Determine which identified impacts
ensure a consistent and integrated assessment, or are significant, based on their likelihood and
explain why there are differences in the assessment expected magnitude.
periods.
• Step 4. Determine the nature of the change
(positive or negative).

7.3 Characterize each specific impact • Step 5. Report the results.

in terms of likelihood, magnitude
and nature of the change
7.3.1 Step 1: Assess the likelihood that each
It is a key recommendation to characterize each sustainable development impact will occur
specific impact identified in Chapter 6 based on:
For each sustainable development impact identified
• the likelihood that each impact will occur in Chapter 6, users should assess the likelihood that
it will occur by classifying each impact according to
• the magnitude of each impact the options in Table 7.2. For ex-ante assessments,
this involves predicting the likelihood of each impact
• the nature of the change (positive or negative). occurring in the future as a result of the policy.
For ex-post assessments, it involves assessing the
Based on the assessment of likelihood and likelihood that the impact occurred in the past as a
magnitude, it is a key recommendation to determine result of the policy, since impacts may have occurred
which identified impacts are significant, in during the assessment period for reasons unrelated
consultation with stakeholders. Assessing the to the policy being assessed. If a given impact is
significance of each specific impact is an important unlikely to occur, the impacts that follow from that
step for the qualitative assessment. It is also useful impact can also be considered unlikely to occur. If
when identifying the specific impacts to be included users cannot determine the likelihood of a specific
in the quantitative assessment boundary, where impact, it should be classified as “possible”.
significance is used to determine which impacts
should be quantified (in Section 8.1).

The following steps can be used to characterize each

specific impact:

• Step 1. Assess the likelihood that each

sustainable development impact will occur.
74 Sustainable Development Methodology

TABLE 7.2

Assessing likelihood of sustainable development impacts

Approximate
likelihood
Likelihood Description (rule of thumb)

Very likely Reason to believe the impact will happen (or did happen) as a result of the ≥90%
policy.

Likely Reason to believe the impact will probably happen (or probably happened) as a <90% and ≥66%
result of the policy.

Possible Reason to believe the impact may or may not happen (or may or may not have <66% and ≥33%
happened) as a result of the policy. About as likely as not. Cases where the
likelihood is unknown or cannot be determined should be considered possible.

Unlikely Reason to believe the impact probably will not happen (or probably did not <33% and ≥10%
happen) as a result of the policy.

Very unlikely Reason to believe the impact will not happen (or did not happen) as a result of <10%
the policy.

Source: Adapted from WRI (2014).

To the extent possible, the likelihood classification It is not necessary to accurately calculate the relative
should be based on evidence, such as published magnitude of sustainable development impacts at
studies on similar policies and impact categories in this stage, but the classification should be based
the same or other jurisdictions, prior experience, on evidence, to the extent possible. Evidence may
modelling results, risk management methods, include published studies on similar policies and
life cycle assessment (LCA) databases and impact categories in the same or other jurisdictions,
studies, relevant media reports, consultation with prior experience, modelling results, LCA databases
stakeholders, and expert judgment. and studies, relevant media reports, consultation
with experts and stakeholders, and expert judgment.
Users can conduct other types of qualitative Appendix C provides an overview of qualitative
studies, including longitudinal impact assessment, research methods.
sampling, interviews and ethnography, to inform the
assessment. Appendix C provides an overview of If no data or evidence exist to estimate relative
qualitative research methods. magnitudes, expert judgment and stakeholder
consultation should be used to classify impacts as
Because the determination can be subjective, users major, moderate or minor. If this is not possible,
should solicit multiple viewpoints and consult users should classify a given impact as “uncertain” or
stakeholders when assessing the likelihood of “cannot be determined”.
impacts. The ICAT Stakeholder Participation Guide
(Chapter 8) provides more information on how to Magnitude represents the degree of change
consult with stakeholders. resulting, or expected to result, from the policy.
Conceptually, the degree of change should be
characterized relative to a baseline scenario that
7.3.2 Step 2: Assess the expected magnitude represents the events or conditions that would most
of each sustainable development impact likely occur in the absence of the policy. Since this is
a qualitative assessment, this step does not require a
Next, users should classify the magnitude of each detailed baseline assessment.
sustainable development impact as major, moderate
or minor (see Table 7.3).
 Part III: Qualitative approach to impact assessment 75

TABLE 7.3

Estimating relative magnitude of sustainable development impacts

Relative magnitude Description

Major The change in the impact category is (or is expected to be) substantial in size (either positive
or negative).a The impact significantly influences the effectiveness of the policy with respect
to that impact category.

Moderate The change in the impact category is (or is expected to be) moderate in size (either positive
or negative).a The impact somewhat influences the effectiveness of the policy with respect to
that impact category.

Minor The change in the impact category is (or is expected to be) insignificant in size (either positive
or negative).a The impact is inconsequential to the effectiveness of the policy with respect to
that impact category.

Source: Adapted from WRI (2014).

a
The magnitude of the change should be considered relative to the broader conditions relating to the impact category or to the
maximum potential impact from policy options considered feasible.

When determining the magnitude of the change, meaningful results based on the specific context and
it may be useful to consider the extent of the area circumstances.
affected by the policy, such as:
In general, users should assess the magnitude
• a single site (e.g. the impacts are restricted to of each impact relative to the broader conditions
areas within the boundaries of the site) relating to a given impact category (e.g. total level
of air pollution in a region or total number of jobs),
• local impacts (e.g. affecting the water supplies rather than in comparison with other impacts
of a local community) resulting from the policy.

• regional impacts (e.g. affecting habitat areas Users can also classify impacts as major, moderate
that support species of regional significance) or minor in relation to the maximum level of impact
considered feasible from various policy options
• national impacts available in a jurisdiction (e.g. the maximum level of
air quality improvement or job creation considered
• international impacts. feasible and realistic). Users should report the
approaches and reference points used to determine
It may be useful to consider the duration of the the magnitude of impacts.
change in terms of the length of time over which
impacts may occur, such as short term (up to 5 For example, a solar PV incentive policy may have
years), medium term (5–15 years) and long term three impacts in the impact category of air quality.
(greater than 15 years). Each impact should be assessed relative to the
broader conditions – absolute levels of air pollution
It may also be useful to consider the size of the in the region – to determine whether it is minor,
groups (e.g. businesses or consumers) affected by moderate or major. The determination of magnitude
the policy and the scale of change in the underlying can alternatively be in relation to the maximum
activities (e.g. change in vehicle kilometres travelled level of air pollution reduction considered feasible
or electricity consumption). from various policy options that are available. See
Box 7.1 for an example. Note that impacts should be
Determining whether an impact is major, moderate compared based on their absolute value, regardless
or minor requires comparing the expected of whether each impact is increasing or decreasing.
impact with a reference point. Users should
choose a reference point that produces the most
76 Sustainable Development Methodology

BOX 7.1

Example of using estimates to assess relative magnitude of impact for a solar PV incentive policy

A solar PV incentive policy has multiple impacts on the impact category of air quality, as measured by the indicator of sulfur
dioxide (SO2) emissions. These include (1) reduced SO2 emissions from fossil fuel combustion at power plants (assumed to
be approximately 5,000 kg/year), (2) reduced SO2 emissions from extraction and transportation of fossil fuels (assumed to
be approximately 2,000 kg/year) and (3) increased SO2 emissions from extraction and transportation of materials associated
with solar panels (assumed to be approximately 200 kg/year).
Users should first decide the reference point to be used. In this case, the user decides to use the maximum potential impact
from policy options considered feasible as the reference point, and estimates that quantity to be approximately 50,000 kg/
year. Next, the user compares the approximate magnitude of each impact in relation to the reference point. The relative
magnitude of “reduced SO2 emissions from fossil fuel combustion” is 10% (5,000 divided by 50,000), the relative magnitude
of “reduced SO2 emissions from extraction and transportation of fossil fuels” is 4% (2,000 divided by 50,000), and the relative
magnitude of “increased SO2 emissions from extraction and transportation of materials associated with solar panels” is 0.4%
(200 divided by 50,000). Based on this estimation, the first impact is considered major, the second impact is considered
moderate and the third impact is considered minor.

7.3.3 Step 3: Determine which identified 7.3.4 Step 4: Determine the nature of the
impacts are significant, based on their change (positive or negative)
likelihood and expected magnitude
Users should characterize each sustainable
Once the likelihood and magnitude of each impact development impact identified in Chapter 6 as
have been determined, users should combine the positive, negative or neutral. For example, an
scores on likelihood and magnitude to determine increase in available habitat area for a key species
whether each impact is significant. In general, users would be classified as positive, whereas habitat loss
should consider impacts to be significant unless would be considered negative. The determination
they are either minor in size, or unlikely or very should be based on the perspectives of the user,
unlikely to occur (see Figure 7.2). Depending on the policymakers and affected stakeholders. If it is not
context and assessment objectives, users can adopt possible to determine whether the net impact is
other approaches to determining the significance positive or negative, users should classify the impact
of impacts, such as considering unlikely impacts as “unknown” or “cannot be determined”.
that are major or moderate to be significant. Users
should use a consistent approach to determining
significance across all impacts. Both positive and 7.3.5 Step 5: Report the results
negative impacts should be considered equally
significant based on the same likelihood and Users should report the outcomes of the qualitative
magnitude criteria, to avoid a bias towards either assessment for each specific impact – that is, the
positive or negative impacts. Users can separately likelihood, relative magnitude and nature of the
assess positive impacts and negative impacts. change, and whether each impact is significant – and
the methods and sources used. Table 7.5 provides a
reporting template that can be used.

Box 7.2 provides a case study of consulting

stakeholders during the qualitative assessment
process.
 Part III: Qualitative approach to impact assessment 77

FIGURE 7.2

Recommended approach for determining significance, based on likelihood and magnitude

Magnitude

Relative magnitude Minor Moderate Major

Very likely

Likely Significant

Possible

Unlikely Insignificant

Very unlikely

Source: Adapted from WRI (2014).

BOX 7.2

Using stakeholder consultation to qualitatively assess impacts in Malawi

The Initiative for Climate Action and Development in Malawi applied the ICAT Sustainable Development Methodology to
assess the impacts of the Farmer Field Schools Approach, an element of the Malawi National Climate Change Management
Policy. The project was an ex-post assessment of the environmental, social and economic impacts of a group of initiatives
addressing pesticide risk reduction, poverty alleviation, the mainstreaming of climate change impacts in the irrigation sector,
agricultural productivity and diversification, value chain and business development, and governance.
The objective was to assess policy effectiveness by determining whether actions are being implemented as planned and
delivering intended results across multiple impact categories and across different groups in society. The findings will be used
to improve policy design and implementation.
The impact categories, specific impacts and indicators assessed were drawn from the National Climate Change Management
Policy, the objectives of programme donors, and selected indicators from the SDGs. Because of a lack of quantitative data,
the project team carried out a qualitative assessment, using a mixed methods approach of literature review, case studies
and stakeholder consultation.
The project team developed assessment questionnaires that included all the identified impact categories, specific impacts
and indicators. Respondents were asked to qualitatively assess the impacts for each indicator in terms of likelihood,
magnitude, positive or negative impact, and whether the impact was significant. Interviews and focus groups with identified
stakeholders were carried out by enumerators who had completed training specifically for this project.
Target groups of stakeholders for the interviews were district government officials, representatives from non-governmental/
civil society organizations, and community stakeholders (mostly participants in the Farmer Field Schools). Care was taken to
ensure that marginalized groups were included in the consultation process. To identify community stakeholders, the project
leads consulted the National Youth Network on Climate Change, the Coalition of Women Farmers and the Federation of
Disability Organizations in Malawi. In total, 401 people were engaged, of whom 203 responded; respondents were evenly
distributed across regions and groups of stakeholders.
Table 7.4 provides examples of qualitative assessment results from the stakeholder respondents.
78 Sustainable Development Methodology

BOX 7.2, continued

Using stakeholder consultation to qualitatively assess impacts in Malawi

TABLE 7.4

Examples of stakeholder responses for one programme

Dimension Summary of stakeholder responses

Environmental • Water, land and waste impacts were considered to be likely, of major magnitude, positive and
impacts significant.
• Water acidification was considered to be very likely, of major magnitude, significant and negative.

Social impacts • Health and well-being, education and culture, and welfare and equality indicators were
considered to be likely, of major magnitude, positive and significant.
• Institutions and laws, indicators of public participation in policymaking, and access to
administrative and judicial remedies were considered to be likely, of only moderate impact and
positive.
• Labour rights and youth labour conditions were considered to be unlikely and not significant.
• Quality of jobs and fairness of wages were considered not applicable by the respondents.

Economic • Jobs, wages and worker productivity indicators were marked as not applicable by respondents.
impacts • Business and technology, growth in new sustainable industries, and innovation were considered
to be very likely, of major magnitude, positive and significant.

The results included a recommendation to introduce a quantitative aspect to performance measurement in the future,
which can be used to define objectives, measure baseline data and track performance through a database.

7.4 Summarize the qualitative the results are mixed and the conclusion is not clear
assessment results for each impact for a given impact category, users should provide a
category balanced summary that includes both positive and
negative impacts. See Table 7.5 for an example of
As the last step of the qualitative assessment, it is summarizing the qualitative assessment results.
a key recommendation to summarize the qualitative
assessment results for each impact category, taking It is a key recommendation to separately assess the
into account all significant impacts. This involves impacts of the policy on different groups in society,
summarizing the net impact of the policy on each where relevant. If relevant and feasible, users
impact category in descriptive terms, based on the should separately summarize the conclusions for
qualitative assessment of specific impacts. in-jurisdiction and out-of-jurisdiction impacts. Users
should consult stakeholders when summarizing the
Users should comprehensively consider all assessment results to ensure that the qualitative
significant impacts within each impact category, summary properly characterizes the impact for
taking into account the magnitude and likelihood of each impact category. Stakeholders should be
both positive and negative impacts, and provide a informed about the methods and sources used to
succinct summary of the qualitative results for each determine the likelihood and magnitude of impacts.
impact category. Users should conclude that the If insignificant impacts are deemed important
policy has an overall positive or negative impact on by stakeholders, users should acknowledge the
a given impact category if the assessment of each existence of such impacts in the summary.
significant impact is either positive or negative. If
 Part III: Qualitative approach to impact assessment 79

TABLE 7.5

Reporting the qualitative assessment results for a solar PV incentive policy

Chapter 5 Chapter 6 (identify specific impacts)

Impact categories In- or Type of

included in the out-of- impacts
assessment Specific impacts identified jurisdiction (optional)

Climate change Reduced GHG emissions from grid-connected fossil fuel– In

mitigation based power plants

Reduced GHG emissions from distributed fossil fuel In

generation

Reduced GHG emissions associated with manufacturing of In

new fossil fuel generation plants

Reduced GHG emissions from fossil fuel extraction and Both

transportation

Increased GHG emissions from solar production, Both

transportation and installation

Increased GHG emissions from increased production of In

goods and services due to increased income

Air quality/health Reduced air pollution from grid-connected fossil fuel–based In

impacts of air power plants
pollution
Reduced air pollution from distributed fossil fuel generation In

Reduced indoor air pollution from traditional use of biomass In

Reduced air pollution from manufacturing of new fossil fuel In

generation plants

Reduced air pollution from fossil fuel extraction and Both

transportation

Increased air pollution from solar PV production, Both

transportation and installation

Increased air pollution from increased production of goods In

and services due to increased income

Waste generation Decreased waste generation and disposal from reduced In

and disposal fossil fuel generation (e.g. coal ash)

Decreased waste generation and disposal from reduced Both

fossil fuel production and transportation

Increased waste generation and disposal from increased Both

solar production (e.g. silicon tetrachloride waste)

Increased waste generation and disposal from discarded In

solar panels (e.g. cadmium and tellurium)

Energy Increased renewable energy generation from increased solar In

generation

Access to clean, Increased access to clean, affordable and reliable electricity In

affordable and
reliable energy Decreased access to electricity due to fewer new coal power In
plants
80 Sustainable Development Methodology

TABLE 7.5, continued

Reporting the qualitative assessment results for a solar PV incentive policy

Chapter 5 Chapter 6 (identify specific impacts)

Impact categories In- or Type of

included in the out-of- impacts
assessment Specific impacts identified jurisdiction (optional)

Capacity, skills Increase in training for skilled workers in solar-relevant Both

and knowledge sectors
development
Decrease in training for skilled workers in fossil fuel sectors Both

Quality and safety of Increased safety and working conditions due to more jobs Both
working conditions in the solar installation sector, where workers have better
working conditions

Increased safety and working conditions due to fewer jobs Both

in the coal sector, where workers have worse working
conditions

Decreased safety and working conditions due to more jobs Both

in silica mining and solar cell manufacturing, where workers
have worse working condition (e.g. the lung disease silicosis,
exposure to hydrofluoric acid and cadmium)

Jobs Increased jobs in the solar installation, operations and In

maintenance sectors

Increased jobs in the solar panel manufacturing sector Both

Increased jobs in the solar and grid technology sectors, and Both
mining of rare earth minerals for solar cells

Decreased jobs in the fossil fuel power operations and In

maintenance sectors

Decreased jobs in fossil fuel sectors Both

Decreased jobs in fossil fuel generation technology sectors Both

(e.g. supercritical and ultra-supercritical generation)

Income Increased income for households, institutions and other In

organizations due to reduction in energy costs

New business Increased business opportunities for solar manufacturing, Both

opportunities mining, transportation, solar power plants and grid-
associated technologies

Decreased business opportunities for fossil fuel extraction, Both

transportation, fossil fuel power plants, and fossil fuel–
generated associated technologies

Energy Increased energy independence from reduced imports of In

independence fossil fuels

Decreased energy independence from foreign control over In

scarce resources needed to manufacture solar panels
 Part III: Qualitative approach to impact assessment 81

TABLE 7.5, part II

Reporting the qualitative assessment results for a solar PV incentive policy

Chapter 5 Chapter 7 (Qualitatively assess impacts)

Summary of
Impact qualitative
categories Positive assessment
included Specific or results for
in the impacts negative each impact Methods/
assessment identified Likelihood Magnitude impact Significant? category sources used

Climate Reduced GHG Very likely Major Positive Yes Major positive Stakeholder
change emissions from impact from consultation
mitigation grid-connected displacing fossil
fossil fuel–based fuel electricity
power plants with solar
electricity.
Reduced GHG Unlikely Moderate Positive No Although Reference:
emissions from negative Timmons (2012)
distributed fossil impacts do
fuel generation exist, they are
insignificant.
Reduced GHG Unlikely Minor Positive No Stakeholder
emissions consultation
associated with
manufacturing
of new fossil
fuel generation
plants

Reduced GHG Possible Moderate Positive Yes Reference: Clear

emissions Air Task Force
from fossil fuel (2001)
extraction and
transportation

Increased GHG Likely Minor Negative No Reference:

emissions from Mulvaney (2014)
solar production,
transportation
and installation

Increased GHG Likely Minor Negative No Reference:

emissions Druckman and
from increased Jackson (2008)
production
of goods and
services due
to increased
income
82 Sustainable Development Methodology

TABLE 7.5, part II – continued

Reporting the qualitative assessment results for a solar PV incentive policy

Chapter 5 Chapter 7 (Qualitatively assess impacts)

Summary of
Impact qualitative
categories Positive assessment
included Specific or results for
in the impacts negative each impact Methods/
assessment identified Likelihood Magnitude impact Significant? category sources used

Air quality/ Reduced air Very likely Major Positive Yes Major positive Stakeholder
health impacts pollution from impact from consultation
of air pollution grid-connected displacing fossil
fossil fuel–based fuel electricity
power plants with solar
electricity.
Reduced air Unlikely Major Positive No Although Stakeholder
pollution from negative consultation
distributed fossil impacts do
fuel generation exist, they are
insignificant.
Reduced indoor Very likely Major Positive Yes Reference:
air pollution Fullerton, Bruce
from traditional and Gordon
use of biomass (2008)

Reduced air Likely Minor Positive No Expert judgment

pollution from
manufacturing
of new fossil
fuel generation
plants

Reduced air Possible Moderate Positive Yes Reference: Clear

pollution from Air Task Force
fossil fuel (2001)
extraction and
transportation

Increased Likely Minor Negative No Reference:

air pollution Mulvaney (2014)
from solar PV
production,
transportation
and installation

Increased air Likely Minor Negative No Reference:

pollution from Druckman and
increased Jackson (2008)
production
of goods and
services due
to increased
income
 Part III: Qualitative approach to impact assessment 83

TABLE 7.5, part II – continued

Reporting the qualitative assessment results for a solar PV incentive policy

Chapter 5 Chapter 7 (Qualitatively assess impacts)

Summary of
Impact qualitative
categories Positive assessment
included Specific or results for
in the impacts negative each impact Methods/
assessment identified Likelihood Magnitude impact Significant? category sources used

Waste Decreased waste Very likely Moderate Positive Yes Major positive Reference: Clear
generation generation and impacts from Air Task Force
and disposal disposal from reducing fossil (2001)
reduced fossil fuel extraction,
fuel generation transportation
(e.g. coal ash) and
consumption,
Decreased waste Very likely Major Positive Yes which outweigh Reference: Clear
generation moderate or Air Task Force
and disposal insignificant (2001)
from reduced negative
fossil fuel impacts from
production and solar-related
transportation mining and
solar panel
Increased waste Likely Moderate Negative Yes Reference:
disposal
generation and Mulvaney (2014)
disposal from
increased solar
production
(e.g. silicon
tetrachloride
waste)

Increased waste Possible Minor Positive No Reference:

generation and Mulvaney (2014)
disposal from
discarded solar
panels
(e.g. cadmium
and tellurium)

Energy Increased Very likely Major Positive Yes Major positive Stakeholder
renewable impact from consultation
energy increase in
generation from solar electricity
increased solar
generation
84 Sustainable Development Methodology

TABLE 7.5, part II – continued

Reporting the qualitative assessment results for a solar PV incentive policy

Chapter 5 Chapter 7 (Qualitatively assess impacts)

Summary of
Impact qualitative
categories Positive assessment
included Specific or results for
in the impacts negative each impact Methods/
assessment identified Likelihood Magnitude impact Significant? category sources used

Access Increased Very likely Major Positive Yes Major positive Stakeholder
to clean, access to clean, impact from consultation
affordable and affordable increased solar
reliable energy and reliable electricity,
electricity which
outweighs
Decreased Unlikely Minor Negative No unlikely, Stakeholder
access to insignificant consultation
electricity due to negative
fewer new coal impact
power plants

Capacity, Increase in Likely Major Positive Yes Major positive Stakeholder

skills and training for impact from consultation
knowledge skilled workers solar sectors.
development in solar-relevant Although
sectors a negative
impact exists, it
Decrease in Possible Minor Negative No is insignificant. Stakeholder
training for consultation
skilled workers
in fossil fuel
sectors

Quality and Increased safety Very likely Major Positive Yes Major positive Stakeholder
safety of and working impact from consultation
working conditions due solar sectors.
conditions to more jobs Although
in the solar negative
installation impacts exist,
sector, where they are
workers have insignificant.
better working
conditions

Increased safety Likely Moderate Positive Yes Reference: Clear

and working Air Task Force
conditions due (2001)
to fewer jobs
in the coal
sector, where
workers have
worse working
conditions
 Part III: Qualitative approach to impact assessment 85

TABLE 7.5, part II – continued

Reporting the qualitative assessment results for a solar PV incentive policy

Chapter 5 Chapter 7 (Qualitatively assess impacts)

Summary of
Impact qualitative
categories Positive assessment
included Specific or results for
in the impacts negative each impact Methods/
assessment identified Likelihood Magnitude impact Significant? category sources used

Quality and Decreased safety Unlikely Moderate Negative No Major positive Reference: Sarkar
safety of and working impact from (2016)
working conditions due solar sectors.
conditions, to more jobs Although
continued in silica mining negative
and solar cell impacts exist,
manufacturing, they are
where workers insignificant,
have worse continued
working
condition
(e.g. the lung
disease silicosis,
exposure to
hydrofluoric acid
and cadmium)

Jobs Increased jobs Very likely Major Positive Yes Major positive Reference: Solar
in the solar impacts from Foundation (2016)
installation, solar power
operations and plants and
maintenance solar panel
sectors sectors, which
outweigh
moderate
negative
impact on coal
extraction,
transportation
and import/
export sectors

Increased jobs in Very likely Major Positive Yes Reference: Solar

the solar panel Foundation (2016)
manufacturing
sector

Increased jobs Possible Minor Positive No Stakeholder

in the solar and consultation
grid technology
sectors, and
mining of rare
earth minerals
for solar cells
86 Sustainable Development Methodology

TABLE 7.5, part II – continued

Reporting the qualitative assessment results for a solar PV incentive policy

Chapter 5 Chapter 7 (Qualitatively assess impacts)

Summary of
Impact qualitative
categories Positive assessment
included Specific or results for
in the impacts negative each impact Methods/
assessment identified Likelihood Magnitude impact Significant? category sources used

Jobs, Decreased jobs Likely Minor Negative No Major positive Stakeholder

continued in the fossil impacts from consultation
fuel power solar power
operations and plants and
maintenance solar panel
sectors sectors, which
outweigh
Decreased jobs Likely Moderate Negative Yes moderate Stakeholder
in fossil fuel negative consultation
sectors impact on coal
extraction,
Decreased Unlikely Moderate Negative No Stakeholder
transportation
jobs in fossil consultation
and import/
fuel generation
export sectors,
technology
continued
sectors (e.g.
supercritical
and ultra-
supercritical
generation)

Income Increased Very likely Major Positive Yes Major positive Stakeholder
income for impact from consultation
households, savings
institutions on energy
and other spending
organizations
due to reduction
in energy costs

New business Increased Very likely Major Positive Yes Major positive Reference:
opportunities business impact from ConnectAmericas
opportunities solar sectors. (no date)
for solar Although
manufacturing, a negative
mining, impact exists, it
transportation, is insignificant.
solar power
plants and
grid-associated
technologies
 Part III: Qualitative approach to impact assessment 87

TABLE 7.5, part II – continued

Reporting the qualitative assessment results for a solar PV incentive policy

Chapter 5 Chapter 7 (Qualitatively assess impacts)

Summary of
Impact qualitative
categories Positive assessment
included Specific or results for
in the impacts negative each impact Methods/
assessment identified Likelihood Magnitude impact Significant? category sources used

New business Decreased Likely Minor Negative No Major positive Stakeholder

opportunities, business impact from consultation
continued opportunities solar sectors.
for fossil fuel Although
extraction, a negative
transportation, impact exists, it
fossil fuel power is insignificant,
plants, and fossil continued
fuel–generated
associated
technologies

Energy Increased energy Very likely Major Positive Yes Major positive Stakeholder
independence independence impact from consultation
from reduced decreased
imports of fossil fossil fuel
fuels import.
Although
Decreased Possible Minor Negative No a negative Reference:
energy impact exists, it Simmons (2016)
independence is insignificant.
from foreign
control over
scarce resources
needed to
manufacture
solar panels
88 Sustainable Development Methodology

TABLE 7.5, part III

Reporting the qualitative assessment results for a solar PV incentive policy

Chapter 5 Chapter 8 (Define the quantitative assessment boundary)

Impact Included
categories in the
included Feasible quantitative Justification for
in the to assessment exclusions or
assessment Specific impacts identified quantify? boundary? other comments

Climate Reduced GHG emissions from grid- Yes Yes Included

change connected fossil fuel–based power plants
mitigation
Reduced GHG emissions from distributed No No Impact not significant
fossil fuel generation

Reduced GHG emissions associated with - No Impact not significant

manufacturing of new fossil fuel generation
plants

Reduced GHG emissions from fossil fuel No No No reliable data/

extraction and transportation methods available

Increased GHG emissions from solar - No Impact not significant

production, transportation and installation

Increased GHG emissions from increased - No Impact not significant

production of goods and services due to
increased income

Air quality/ Reduced air pollution from grid-connected Yes Yes Included
health impacts fossil fuel–based power plants
of air pollution
Reduced air pollution from distributed fossil No No Impact not significant
fuel generation

Reduced indoor air pollution from traditional No No No reliable data/

use of biomass methods available

Reduced air pollution from manufacturing of No No Impact not significant

new fossil fuel generation plants

Reduced air pollution from fossil fuel No No No reliable data/

extraction and transportation methods available

Increased air pollution from solar PV - No Impact not significant

production, transportation and installation

Increased air pollution from increased - No Impact not significant

production of goods and services due to
increased income
 Part III: Qualitative approach to impact assessment 89

TABLE 7.5, part III – continued

Reporting the qualitative assessment results for a solar PV incentive policy

Chapter 5 Chapter 8 (Define the quantitative assessment boundary)

Impact Included
categories in the
included Feasible quantitative Justification for
in the to assessment exclusions or
assessment Specific impacts identified quantify? boundary? other comments

Waste Decreased waste generation and disposal No No No reliable data/

generation from reduced fossil fuel generation (e.g. coal methods available
and disposal ash)

Decreased waste generation and disposal No No No reliable data/

from reduced fossil fuel production and methods available
transportation

Increased waste generation and disposal No No No reliable data/

from increased solar production (e.g. silicon methods available
tetrachloride waste)

Increased waste generation and disposal No No Impact not significant

from discarded solar panels (e.g. cadmium
and tellurium)

Energy Increased renewable energy generation from Yes Yes Included

increased solar generation

Access Increased access to clean, affordable and Yes Yes Included

to clean, reliable electricity
affordable and
reliable energy Decreased access to electricity due to fewer - No Impact not significant
new coal power plants

Capacity, Increase in training for skilled workers in Yes Yes Included

skills and solar-relevant sectors
knowledge
development Decrease in training for skilled workers in - No Impact not significant
fossil fuel sectors

Quality and Increased safety and working conditions No No No reliable data/

safety of due to more jobs in the solar installation methods available
working sector, where workers have better working
conditions conditions

Increased safety and working conditions No No No reliable data/

due to fewer jobs in the coal sector, where methods available
workers have worse working conditions

Decreased safety and working conditions - No Impact not significant

due to more jobs in silica mining and solar
cell manufacturing, where workers have
worse working condition (e.g. the lung
disease silicosis, exposure to hydrofluoric
acid and cadmium)
90 Sustainable Development Methodology

TABLE 7.5, part III – continued

Reporting the qualitative assessment results for a solar PV incentive policy

Chapter 5 Chapter 8 (Define the quantitative assessment boundary)

Impact Included
categories in the
included Feasible quantitative Justification for
in the to assessment exclusions or
assessment Specific impacts identified quantify? boundary? other comments

Jobs Increased jobs in the solar installation, Yes Yes Included

operations and maintenance sectors

Increased jobs in the solar panel Yes Yes Included

manufacturing sector

Increased jobs in the solar and grid - No Impact not significant

technology sectors, and mining of rare earth
minerals for solar cells

Decreased jobs in the fossil fuel power - No Impact no significant

operations and maintenance sectors

Decreased jobs in fossil fuel sectors Yes Yes Included

Decreased jobs in fossil fuel generation - No Impact no significant

technology sectors (e.g. supercritical and
ultra-supercritical generation)

Income Increased income for households, Yes Yes Included

institutions and other organizations due to
reduction in energy costs

New business Increased business opportunities for solar No No No reliable data/

opportunities manufacturing, mining, transportation, methods available
solar power plants and grid-associated
technologies

Decreased business opportunities for fossil No No Impact not significant

fuel extraction, transportation, fossil fuel
power plants, and fossil fuel–generated
associated technologies

Energy Increased energy independence from Yes Yes Included

independence reduced imports of fossil fuels

Decreased energy independence from - No Impact not significant

foreign control over scarce resources
needed to manufacture solar panels

Abbreviation: -, not applicable

PART IV
Quantitative approach to impact
­assessment
8 Estimating the baseline

This chapter is relevant for users who are following Checklist of key recommendations
the quantitative approach to impact assessment.
Quantifying impacts by defining changes relative to a • Include all significant impacts in the
baseline scenario may not always be necessary to meet quantitative assessment boundary, where
the stated objectives of the assessment. Users can assess feasible
impacts qualitatively (in Chapter 7) or track trends in • Define one or more appropriate indicators
key indicators over time (in Chapter 12). Attributing for each impact category included in the
impacts to specific policies relative to a baseline quantitative assessment boundary
scenario is valuable since it enables an understanding • Define the assessment period
of how effective policies are, relative to what would have • Define a baseline scenario that represents the
happened in the absence of the policy. This information conditions most likely to occur in the absence
enables users to meet a wider range of objectives, of the policy for each indicator included in the
outlined in Chapter 2, such as improving policy design, assessment boundary
selection and implementation, and determining whether • Estimate baseline values over the assessment
policies have been effective. period for each indicator included in the
assessment boundary
The baseline scenario represents the events or • Separately estimate baseline values for
conditions that would most likely occur in the absence of different groups in society, where relevant
the policy being assessed. Properly estimating baseline
values is a critical step, since it has a direct effect on the
estimated impacts of the policy. In this chapter, users
estimate baseline values for each indicator included in 8.1 Define the quantitative
the quantitative assessment boundary. This chapter is assessment boundary and period
relevant to both ex-ante and ex-post assessment, and
provides guidance on estimating ex-ante and ex-post The quantitative assessment boundary defines
baseline scenarios. the scope of the quantitative assessment in terms
of the range of dimensions, impact categories,
specific impacts and indicators that are included in
the quantitative assessment and estimated. Not all
specific impacts identified in Chapter 6 need to be
estimated. It is a key recommendation to include all
significant impacts in the quantitative assessment
boundary, where feasible.

FIGURE 8.1

Overview of steps in the chapter

Define the quantitative Define the baseline scenario

Choose assessment method
assessment boundary and and estimate baseline values
for each indicator
period for each indicator
(Section 8.2)
(Section 8.1) (Section 8.3)
 Part IV: Quantitative approach to impact assessment 93

8.1.1 Choose which specific impacts to in Chapter 5 may need to be revisited based
quantify on the outcomes of Chapters 6 and 7, since the
choice of indicators should be informed by which
Users should determine which specific impacts to specific impacts are significant and included in the
include in the quantitative assessment boundary and quantitative assessment boundary.
estimate, based on:
Users can define one or more indicators for each
• the significance of each impact, as determined impact category. For example, within the impact
in Section 7.3, based on a combination of category of air quality, a user may estimate the
likelihood and magnitude impact of the policy on multiple indicators, such as
particulate matter (PM2.5, PM10), SO2 and nitrogen
• the feasibility of estimating each impact. oxides (NOx).

Feasibility may depend on data availability, technical Some indicators for a given impact category are likely
capacity and resources available to estimate impacts, to be more feasible to quantify than others. Users
or other factors. If it is not feasible to estimate should choose indicators for which it is possible to
certain impacts, the decision to exclude them from collect data and quantify impacts. If it is not possible
the quantitative assessment boundary should be to quantify a particular indicator, users should either
explained and justified. Table 7.5 provides a template select a different indicator for the same impact
that can be used to report whether it is feasible to category or qualitatively assess any indicators and
quantify each significant impact, whether the impact specific impacts that cannot be quantified.
is included in the quantitative assessment boundary
and, if it is not included, a justification for exclusion. The indicators selected in this step will be
The example in Table 7.5 shows that, out of many estimated in the baseline and policy scenarios
identified impacts, 10 specific impacts are included in (in Chapters 8–10), and monitored over time
the quantitative assessment boundary. This short list (Chapter 12). Table 8.1 presents indicators selected
of specific impacts is presented in Table 8.1. for a solar PV incentive policy.

In general, users should not exclude any impacts

from the quantitative assessment boundary 8.1.3 Define the assessment period
that would compromise the relevance of the
overall assessment. Users should ensure that the It is a key recommendation to define the assessment
assessment appropriately reflects the impacts period. In general, the assessment period for a
resulting from the policy and that it serves the quantitative assessment should be the same as
decision-making needs of users of the assessment the period defined in Section 7.2 for the qualitative
report. Exclusions may lead to misleading and biased assessment. In some cases, users may want to
results that do not accurately represent the impacts choose a different assessment period for the
of the policy. Where possible, instead of excluding quantitative assessment, based on objectives, data
significant impacts, users should use simplified or availability or other reasons.
less rigorous estimation methods to approximate
each impact, or use proxy data to fill data gaps. Any Box 8.1 provides an example from an assessment in
significant impacts that are not quantified should be Mexico of how the choice of assessment period can
described qualitatively. have a significant impact on the overall assessment
results.

8.1.2 Choose which indicators to quantify

It is a key recommendation to define one or more

appropriate indicators for each impact category
included in the quantitative assessment boundary.
The indicator(s) will be quantified in the baseline
scenario and policy scenario to estimate the impact
of the policy. Each indicator will generally require a
different assessment method.

Section 5.2 introduces indicators and provides

examples in Table 5.5. The initial indicators chosen
94 Sustainable Development Methodology

TABLE 8.1

Example of defining the quantitative assessment boundary for a solar PV incentive policy

Chapter 6 (Identify
Chapter 5 specific impacts) Chapter 8 (Define the quantitative assessment boundary)

Impact Included
categories Specific impacts in the
included included in the quantitative
in the quantitative assessment Feasible to assessment
assessment boundary Indicators to quantify quantify? boundary?

Climate Reduced GHG emissions GHG emissions (tCO2e/year) Yes Yes

change from grid-connected fossil
mitigation fuel–based power plants

Air quality/ Reduced air pollution from Emissions of PM2.5, PM10, SO2 Yes Yes
health impacts grid-connected fossil fuel– and NOx (t/year); number of
of air pollution based power plants deaths due to air pollution

Energy Increased renewable energy Solar installed capacity Yes Yes

generation from increased (MW); % solar of total
solar generation installed capacity; % solar
of total installed capacity of
renewable energy sources

Access Increased access to clean, Number of houses/ Yes Yes

to clean, affordable and reliable buildings/facilities with
affordable and electricity access to clean energy
reliable energy resulting from the policy

Capacity, Increase in training for Number of new skilled Yes Yes

skills and skilled workers in solar- trainees and workers on the
knowledge relevant sectors ground
development

Jobs Increased jobs in the solar Number of new jobs Yes Yes
installation, operations and resulting from the policy
maintenance sectors

Increased jobs in the solar Number of new jobs Yes Yes

panel manufacturing sector resulting from the policy

Decreased jobs in fossil fuel Number of jobs reduced Yes Yes

sectors resulting from the policy

Income Increased income for Savings in annual electric Yes Yes

households, institutions and bills ($/year)
other organizations due to
reduction in energy costs

Energy Increased energy Reduction in coal imports Yes Yes

Independence independence from from the policy (t/year)
reduced imports of fossil
fuels

Abbreviations: MW, megawatt; t, tonne; tCO2e, tonne of carbon dioxide equivalent

 Part IV: Quantitative approach to impact assessment 95

BOX 8.1

Selection of assessment periods and how assessment results vary over different time periods
for a policy in Mexico

A researcher at Aalto University assessed the sustainable development impacts of two climate actions in public buildings
in Mexico: installing PV panels and changing fluorescent lamps to LED lamps. These actions are part of the Carbon
Management Plan of the Mexican state of Jalisco. The assessment illustrates how the impacts of a policy can change over
time. The net impacts of the policy may not be linear, and the nature of impacts could change from negative to positive or
vice versa under different assessment periods. In such cases, it is important to assess and report both short- and long-term
impacts.
Selected results of the assessment are shown in Table 8.2, and Figure 8.2 illustrates the trends in the policy’s net impact
over time for three selected impact categories. The assessment found that the nature and scale of impacts across short-
and long-term time horizons, measured as the percentage of cumulative net impact compared with the baseline scenario,
remain stable for some impact categories (GHG emissions, depletion of fossil resources, and air quality). For others (mineral
resources depletion), the scale of the impact changes dramatically over time. For impact categories such as human toxicity
and water ecotoxicity, the net impact changes from negative to positive when the assessment period is expanded from
5 years to 17 years. The policy had nearly all positive environmental impacts using a longer assessment period, compared
with mixed results using a short assessment period.

TABLE 8.2

Summary of environmental impacts resulting from LED lamp replacement policy

over 5- and 17-year assessment periods

Cumulative impact over 5 years Cumulative impact over 17 years

Impact Baseline Policy Net % net Baseline Policy Net % net

category Unit scenario scenario impact impact scenario scenario impact impact

GHG emissions tCO2e 239 146 Reduction –39 724 409 Reduction –43
of 93 of 315

Depletion kg Cu 66 243 Increase 267 288 315 Increase 9

of mineral eq of 177 of 27
resources

Depletion of kg oil 74,990 46,104 Reduction –39 226,106 128,755 Reduction –43
fossil resources eq of 28,886 of 97,351

Freshwater m3 531 467 Reduction –12 1,851 1,170 Reduction –37

consumption of 64 of 681

Air quality DALY 0.24 0.16 Reduction –34 0.64 0.37 Reduction –42
of 0.08 of 0.27

Human toxicity DALY 0.025 0.029 Increase 15 0.088 0.061 Reduction –30
of 0.004 of 0.027

Water kg 1,4- 6,255 7,190 Increase 15 24,739 18,549 Reduction –25

ecotoxicity DCB of 936 of 6,190

Abbreviations: DALY, disability-adjusted life year; kg 1,4-DCB, kilograms of 1,4-dichlorobenzene; kg Cu eq, kilograms of copper
equivalent; kg oil eq, kilograms of oil equivalent
Note: Positive (good) results are shown in black and negative (bad) results are shown in red.
 96

HUMAN TOXICITY FRESHWATER CONSUMPTION DEPLETION OF FOSSIL FUEL

(DALY) (m 3) RESOURCES (kg oil eq)

FIGURE 8.2
BOX 8.1, continued

–0.01
0.01
0.03
0.05
0.07
0.09
0
250
500
750
1,000
1,250
1,500
1,750
0
30,000
60,000
90,000
120,000
150,000
180,000
210,000
for a policy in Mexico

2014 2014 2014

2014–2015 2014–2015 2014–2015

Baseline scenario
Baseline scenario
Baseline scenario
Sustainable Development Methodology

2014–2016 2014–2016 2014–2016

2014–2017 2014–2017 2014–2017

2014–2018 2014–2018 2014–2018

2014–2019 2014–2019 2014–2019

freshwater consumption and human toxicity

2014–2020 2014–2020 2014–2020

Policy scenario
Policy scenario
Policy scenario
2014–2021 2014–2021 2014–2021

2014–2022 2014–2022 2014–2022

2014–2023 2014–2023 2014–2023

2014–2024 2014–2024 2014–2024

2014–2025 2014–2025 2014–2025

Cumulative impact of the policy on depletion of fossil fuel resources,

2014–2026 2014–2026 2014–2026

2014–2027 2014–2027 2014–2027

2014–2028 2014–2028 2014–2028

2014–2029 2014–2029 2014–2029

2014–2030 2014–2030 2014–2030

2014–2031 2014–2031 2014–2031

Selection of assessment periods and how assessment results vary over different time periods
 Part IV: Quantitative approach to impact assessment 97

8.2 Choose assessment method assessments can use any method. If appropriate,
for each indicator users can use a different assessment method for
each indicator included in the assessment boundary.
Estimating the impacts of a policy involves comparing The choice of method should depend on which
the outcome of the policy with an estimate of what would yield the most accurate results for a given
would most likely have happened in the absence of indicator in the context of the assessment objectives,
that policy. and the data and resources available.

The impact of a policy can be quantified in three

ways: 8.2.1 Scenario method

• Scenario method – comparison of a baseline Using the scenario method, users quantify the
scenario with a policy scenario for the same impact of a policy by comparing two scenarios:
group or region, where separate baseline and
policy scenarios are defined and estimated • the baseline scenario, which represents the
events or conditions most likely to occur in the
• Deemed estimates method – a simplified absence of the policy (or package of policies)
approach to the scenario method, where the being assessed
change resulting from a policy is estimated
directly without separately defining and • the policy scenario, which represents the
estimating baseline and policy scenarios events or conditions most likely to occur in the
presence of the policy (or package of policies)
• Comparison group method – comparison being assessed.
of one group or region affected by the policy
with an equivalent group or region not Figure 8.3 illustrates using the scenario method to
affected by the policy. quantify the impact of a renewable energy policy on
renewable electricity generation.
Ex-ante assessments can only use the scenario
method or deemed estimates method. Ex-post

FIGURE 8.3

Example of scenario method

Renewable energy:
ELECTRICITY GENERATION

policy scenario
RENEWABLE ENERGY
SUPPLY IMPACT OF
THE POLICY

Renewable energy:
baseline scenario

2010 2050
98 Sustainable Development Methodology

In the scenario method, the baseline scenario scenarios. Box 8.2 provides an example of applying
depends on assumptions relating to key impact the scenario method. Appendix A includes examples
drivers over the assessment period. Drivers include of using the scenario method for a solar PV incentive
other policies that have been implemented or policy.
adopted, as well as non-policy drivers, such as
economic conditions, energy prices and technological
development. 8.2.2 Deemed estimates method

Baseline scenarios can be determined ex-ante or The deemed estimates method (sometimes called
ex-post. An ex-ante baseline scenario is a forward- a “deemed savings” or “unit savings” approach) is
looking baseline scenario, typically established a simplified variation of the scenario method. It
before implementation of the policy, which is based involves calculating the impact of a policy without
on forecasts of drivers (such as projected changes in separately defining and estimating baseline and
population or economic activity, or other drivers that policy scenarios and comparing the two. This
affect the impact category), in addition to historical method may be appropriate for certain common or
data. Ex-ante baseline scenarios are used for ex-ante homogeneous policies and actions where deemed
assessment in Chapter 9. estimate values are reliable, or in cases where the
scenario method is not practical.
An ex-post baseline scenario is a backward-looking
baseline scenario established during or after To carry out the approach, users estimate the
implementation of the policy. Ex-post baseline impact by multiplying the number of projects or
scenarios should include updates to the ex-ante measures taken as a result of the policy (such as the
forecasts of drivers, if an ex-ante assessment was number of solar PV systems installed) by deemed
first undertaken. Ex-post baseline scenarios are used estimate values that represent the change per
for ex-post assessment in Chapter 10. project or measure taken (such as the change in
jobs or reduction in air pollution per megawatt of
The methods described in this chapter apply to solar energy installed). For example, to estimate the
both ex-ante and ex-post baseline scenarios. See energy savings from a policy to replace inefficient
Figure 8.4 for an illustration of both types of baseline lightbulbs with energy-efficient lightbulbs, a user can

FIGURE 8.4

Ex-ante and ex-post baseline scenarios

io
cenar
bas eline s
Ex-ante
INDICATOR VALUE

IMPACT OF
POLICY OR
Ex-post baseline scenario
ACTION
IMPACT OF POLICY (EX-ANTE)
Historical
OR ACTION (EX-POST)
values Ex-p
ost
poli
c
(obs y scena
erve rio Ex-an
d) te poli
cy sce
nario

2015 2020 2025

Source: Adapted from WRI (2014).

 Part IV: Quantitative approach to impact assessment 99

BOX 8.2

Scenario method example – waste policy in Brazil

To quantify a range of socioeconomic benefits of an integrated solid waste management policy in Brazil, a baseline scenario
was compared with four policy scenarios. The baseline scenario assumes that, without the policy, 58% of solid waste would
go to sanitary landfills, most of which flare the methane produced. The remaining waste goes to open dumps, where
methane vents to the atmosphere.
The four policy scenarios were as follows:
1. All waste sent to a sanitary landfill, with 50% of landfill gas (LFG) collected and flared.
2. Same as scenario 1, but LFG is used to generate electricity that displaces natural gas from the power grid.
3. Anaerobic digestion of organic waste, with electricity generation.
4. Composting of organic waste.
The calculated impacts of implementing all four policy scenarios together, relative to the baseline scenario, are as follows:
• 44,000–110,000 jobs are created.
• 0.5–1.1% of Brazil’s electricity demand is saved.
• Brazil’s gross domestic product (GDP) increases by $13.3–35.2 billion between 2012 and 2032.
• GHG emissions are reduced by 158–315 MtCO2e.
• 2,500–4,900 premature deaths from air pollution are avoided, with a monetized value of $5.5–10.6 billion
• 550,000–1.1 million tonnes of crops are saved, worth $61–120 million.
• Total net present value of development objectives exceeds $100 billion.

Source: ClimateWorks Foundation and World Bank Group (2014).

multiply the number of lightbulbs replaced by the example, by adjusting the number of hours of
difference in energy use between a typical inefficient operation to represent the local context, or using a
bulb and a typical replacement bulb. conservative estimate where there is uncertainty.
Deemed estimate values can be customized to local
Such approaches simplify the calculation and data circumstances or calculated based on local data,
collection required to quantify the impact of a policy. rather than using default factors.
However, the calculation risks being oversimplified
and inaccurate. The deemed estimates method Users can apply a different method for each
typically holds constant many factors that could indicator being assessed. For example, the deemed
influence the indicator. The estimated impact value estimates method can be used for one indicator and
(or “deemed estimate”) is an implicit representation the scenario method for other indicators. Box 8.3
of the difference between a baseline value and a provides an example of using the deemed estimates
policy scenario value, which may not use accurate method. Appendix A includes examples of using the
or representative baseline or policy scenario deemed estimates method for a solar PV incentive
assumptions. The deemed estimate value may policy.
assume that the maximum impact (such as energy
savings) will be attained, if it does not take into
account the specific conditions under which the 8.2.3 Comparison group method
policy is implemented. For example, using the
lightbulb example, the number of hours each The comparison group method can only be used
lightbulb is in use in the implementing country for ex-post assessments and if an equivalent
may differ from the assumptions taken from comparison group exists. To reliably and credibly
impacts in another country. These factors should implement a comparison group method, actors
be taken into consideration when calculating affected by the policy (the policy group) and actors
impacts to ensure that estimates are realistic – for not affected by the policy (the comparison group or
100 Sustainable Development Methodology

control group) must be otherwise equivalent. Under If an appropriate comparison group is not available,
ideal experimental conditions, the two groups would the scenario method or deemed estimates method
be randomly assigned to ensure that any differences should be used. In some cases, data obtained from
between the groups are a result of the policy, rather a comparison group can also be used to update,
than any underlying systematic differences or calibrate or validate assumptions and data used in
biases. If random assignment is not possible, other the scenario method or deemed estimates method.
methods can be used to control for external factors, Box 8.4 provides an example of the approach.
avoid “selection bias”, and ensure valid comparisons
(described further in Chapter 10).23 The remainder of this chapter focuses on steps
involved in applying the scenario method. Guidance

BOX 8.3

Example of deemed estimates method

A Gold Standard (GS) study used a deemed estimates method to capture and monetize the environmental and
socioeconomic net benefits associated with GS carbon projects. To quantify the improvements in health from a cookstoves
project, the mortality rate was applied to the number of households with cookstoves to determine the reduction in mortality.
First, the indicator was identified as the difference in indoor PM2.5. Next, the study created an index based on the linear
relationship between indoor air quality and mortality. The percentage reduction in mortality was calculated by applying PM2.5
changes to the index. The mortality rate was then applied to the number of households with cookstoves to determine the
reduction in mortality.

Source: Gold Standard (2014).

BOX 8.4

Example of deemed estimates method

The United Kingdom Government provides analysts and policymakers at all levels of government with guidance on how
to assess and review policies and projects to ensure that public funds are well spent. It views evaluation as essential to
determining whether policies are effective.
The guidance, provided in The Magenta Book, includes approaches for using a control group to establish a baseline
(i.e. counterfactual) scenario. It suggests that controlling policy allocation (i.e. which individuals or areas receive policy
interventions, and when) can play a key role in successful impact evaluation by affecting whether there is a meaningful
comparison group. The guidance offers several examples of how to do this:
• Pilots. Allow the policy to be tried and information to be collected before committing full-scale resources. Not every
potential subject is exposed to the policy, and people who are not exposed can act as a control group.
• Randomization and randomized control trials (RCT). Allocate by lottery or other purely random mechanism which
individuals, groups or local areas receive the policy. Carefully conducted, an RCT provides the clearest evidence of whether
a policy has had an impact.
• Phased introduction. Implement the policy sequentially over a period of time. The periods when some participants have
received the intervention and others have not can serve to generate a comparison group.

Source: HM Treasury, United Kingdom (2011).

23
For more information on the applicability of the comparison
group method, see Coalition for Evidence-Based Policy (2014).
 Part IV: Quantitative approach to impact assessment 101

on the comparison group method is provided in methods and data can be used, it is important to
Chapter 10. report the methods, assumptions and data used to
estimate the baseline scenario.

Users can choose different levels of accuracy

8.3 Define the baseline scenario for different impact categories included in the
and estimate baseline values for assessment. Users should consider the resources
each indicator available for each impact category being assessed,
and focus efforts on achieving higher levels of
This section provides guidance on defining the accuracy for impact categories determined to be the
baseline scenario and estimating baseline scenario most relevant and significant. The availability of data,
values using the scenario method. It is applicable to methods and models, or resources may constrain
all ex-ante assessments and to ex-post assessments the level of accuracy, even for high-priority impacts.
that use the scenario method. Users should clearly document the uncertainty –
either qualitatively or quantitatively – associated with
Figure 8.5 outlines the steps in this section. Users the results and explain how the methods chosen
may find it useful to follow the steps in this section for the assessment provide an acceptable level of
separately for each impact category being estimated, accuracy.
since the choices made regarding methods and data
are likely to be different for each impact category. Estimation of the baseline scenario can range
In this case, users should complete the steps for from simple to complex, as explained below and
one impact category at a time, then repeat the illustrated in Figure 8.6:
process for each impact category included in the
assessment. Involving stakeholders in the selection • Constant baseline. A constant baseline uses
and estimation of baseline scenarios is important to historical or current values as the baseline
ensure credible assumptions and valid results. scenario. This assumes that there will be no
change in the impact category in the future
Appendix A provides an example of carrying out the in the absence of the policy. This is a simple
steps in this section for a solar PV incentive policy. “before” and “after” comparison to indicate the
impacts of the policy.

8.3.1 Select a desired level of accuracy • Simple trend baseline. A simple trend
and complexity baseline uses historical trends as the basis for
the baseline scenario, and assumes that the
A range of methods and data can be used to historical trend will remain the same into the
estimate the baseline scenario. In general, users future in the absence of the policy. This can
should follow the most accurate approach that is take the form of a simple linear extrapolation,
feasible in the context of the assessment objectives, exponential extrapolation or other forms of
capacity and resources. Because a wide variety of extrapolation.

FIGURE 8.5

Overview of steps in defining the baseline scenario and estimating baseline scenario values

Define the
Define the
Select a methods and Estimate
most likely
desired level of parameters Collect data for baseline
baseline
accuracy and needed to each indicator values for each
scenario for
complexity estimate (Section 8.3.4) indicator
each indicator
(Section 8.3.1) baseline values (Section 8.3.5)
(Section 8.3.2)
(Section 8.3.3)
102 Sustainable Development Methodology

• Advanced trend baseline. An advanced A constant baseline is the simplest option and may
trend baseline is a more complex approach be appropriate when indicators are considered likely
that models the impact of many interacting to remain stable over time. A simple trend baseline
elements, such as the impacts of non-policy is most appropriate if the change in indicator values
drivers (such as macroeconomic conditions) (rather than actual indicator values) is expected to
and other policies in affecting conditions in remain stable over time. In general, more advanced
the future. baselines are likely to be more accurate, since
they take into account various drivers that affect
The choice of baseline scenario depends on which conditions over time. However, more advanced
is most appropriate for a given impact category and baselines will only be more accurate if the data
situation, as well as users’ resources, capacity, access and methods available to integrate the impacts of
to data, and availability of appropriate models and multiple drivers are robust. Users should weigh
methods. Users should choose methods and data the priority of each impact category and allocate
that yield the most accurate results within a given resources accordingly when determining the
context, based on the methodological and data complexity of the baseline scenario.
options available.

FIGURE 8.6

Examples of constant, simple trend and advanced trend baselines

CONSTANT BASELINE
NUMBER OF JOBS

2015 2020

SIMPLE TREND BASELINE

NUMBER OF JOBS

2014 2015 2016 2020

ADVANCED TREND BASELINE

NUMBER OF JOBS

2014 2015 2016 2020

 Part IV: Quantitative approach to impact assessment 103

8.3.2 Define the most likely baseline • discrete baseline alternatives, practices,
scenario for each indicator technologies or scenarios (such as the least-
cost alternative practice or technology),
A critical step in applying the scenario method identified using environmental, financial,
is to define the baseline scenario. It is a key economic or behavioural analysis or modelling
recommendation to define a baseline scenario that
represents the conditions most likely to occur in the • a performance standard or benchmark that
absence of the policy for each indicator included in indicates baseline trends.
the assessment boundary.
Including other policies
Users should create a baseline scenario for each In addition to the policy being assessed, there
significant impact to be quantitatively assessed, are likely to be other policies, actions or projects
where feasible. The baseline scenarios may be that affect the indicator being estimated. These
developed separately for each impact of interest. may include regulations and standards, taxes
and charges, subsidies and incentives, voluntary
The most likely baseline scenario depends on drivers agreements, information instruments, or other types
that would affect the impact in the absence of the of policies and actions.
policy being assessed. Identifying key drivers for
each significant impact being assessed and making In the case of a national solar PV incentive policy,
reasonable assumptions about their most likely other policies that may affect the amount of solar
values in the absence of the policy being assessed PV installed by households and businesses in the
can have a large effect on the baseline scenario, and baseline scenario include national regulations that
consequently on the eventual estimate of the impact facilitate connection of distributed generation to
of the policy. the electric grid (other national policies), municipal
incentives to promote renewable energy at the local
Drivers that affect baseline values are divided into level (subnational policies), and utility incentives for
two types: solar PV installation (private sector actions). These
other policies affect conditions in the baseline
• other policies – policies, actions and projects, scenario and should be considered when a user is
other than the policy being assessed, that are determining the incremental impact of the national
expected to affect the impacts included in the solar PV policy compared with what would have
assessment boundary happened in the absence of the policy. Appendix A
provides an example of including other policies in the
• non-policy drivers – other conditions, such as baseline scenario.
socioeconomic factors and market forces, that
are expected to affect the impacts included in To identify other policies and actions to consider
the assessment boundary in the baseline scenario, users should identify key
parameters in the assessment – such as the amount
Users should ensure that baseline scenarios defined of solar PV installed – and identify other policies and
for each impact category are consistent. That is, actions that affect the same parameters.
where different impact categories are affected by
common drivers or assumptions, the same values Users should include all other policies, actions and
should be used for the baseline scenarios for each projects in each baseline scenario that:
impact category. For example, if GDP is a common
driver needed for assessing both the job impacts and • have a significant effect on the impacts
the economic developments impacts of a solar PV included in the assessment boundary
incentive policy, users should use the same assumed
GDP values for both impact categories. • are implemented or adopted at the time
the assessment is carried out (for ex-ante
Users should identify plausible baseline options and assessment) or during the assessment period
choose the option that is considered to be the most (for ex-post assessment).
likely to occur in the absence of the policy. The choice
should be made in consultation with stakeholders Table 8.3 provides definitions of implemented,
and experts. Possible options include: adopted and planned policies, and guidance on
whether to include each in the baseline scenario.
• continuation of current technologies, practices
or conditions
104 Sustainable Development Methodology

Published baseline values may already include stakeholders, expert judgment, modelling results, or
the impact of existing policies and actions in the other methods.
baseline scenario. If it is not possible to include a
relevant policy in the baseline scenario, users should In the case of a solar PV incentive policy, non-policy
document and justify its exclusion. drivers that affect the amount of solar PV installed
by households and businesses in the baseline
Users can establish a significance threshold or scenario may include the price of solar PV systems
other criteria to determine which policies, actions (the less expensive they are, the more households
and projects are significant and should be included. and businesses will install them) and the price of
For other policies that are included, users should electricity (the more expensive electricity from the
determine whether they are designed to operate grid is, the greater the incentive for households and
indefinitely or are limited in duration. Users should businesses to install solar PV systems). These factors
assume that policies will operate indefinitely unless affect conditions in the baseline scenario and should
an end date is explicitly stated. be considered to determine the impact of the solar
PV incentive policy compared with what would have
Including non-policy drivers happened in the absence of the policy.
Non-policy drivers include a wide range of exogenous
factors, such as socioeconomic factors and market Users should include all non-policy drivers in the
forces, that may cause changes in the impact baseline scenario that are not caused by the policy
category but are not a result of the policy being being assessed (i.e. that are exogenous to the
assessed. Users should identify non-policy drivers assessment), and that are expected to result in a
based on literature reviews of similar assessments significant change in calculated impacts between the
and policies, consultations with relevant experts and baseline scenario and the policy scenario. In ex-ante

TABLE 8.3

Definitions of implemented, adopted and planned policies and actions

Guidance for inclusion in the

Policy status Definition baseline scenario

Implemented Policies that are currently in effect, as evidenced by Should be included for both ex-ante
one or more of the following: (1) relevant legislation and ex-post assessments.
or regulation is in force, (2) one or more voluntary
agreements have been established and are in force,
(3) financial resources have been allocated, (4) human
resources have been mobilized.

Adopted Policies for which an official government decision has Should be included for ex-ante
been made and there is a clear commitment to proceed assessment if polices are likely to be
with implementation, but implementation has not yet implemented and there is enough
begun (e.g. a law has been passed, but regulations to information to estimate the impacts.
implement the law have not yet been established or are Should not be included for ex-post
not being enforced). assessment.

Planned Policy options that are under discussion, and have a In some cases, users may want to
realistic chance of being adopted and implemented include planned policies for ex-ante
in the future, but have not yet been adopted or assessment – for example, if the
implemented. objective is to assess the impact of
one planned policy relative to other
planned policies.
Should not be included for ex-post
assessment.

Source: Adapted from WRI (2014).

 Part IV: Quantitative approach to impact assessment 105

assessments, users do not need to include drivers make a reasonable assumption about the value of
that are expected to remain the same under both the a parameter. Users may need to use proxy data,
policy scenario and the baseline scenario. Users can interpolate information, estimate a rate of growth, or
establish a significance threshold or other criteria to use other types of assumptions or judgment. Users
determine which non-policy drivers are significant. can apply their own expert judgment or consult
experts. When doing so, it is important to document
To identify non-policy drivers that should be that other data sources were not available, and the
considered in the baseline scenario, users should reasons why, and the rationale for the value chosen.
identify key parameters in the assessment – such as
the amount of solar PV installed – and identify other
policies and actions that affect the same parameters. 8.3.3 Define the methods and parameters
needed to estimate baseline values
Published baseline values may already include the
impact of non-policy drivers in the baseline scenario. For each indicator to be assessed, users should first
If it is not possible to include a relevant non-policy identify a method (such as an equation, algorithm
driver in the baseline scenario, users should or model) for estimating the baseline scenario, then
document and justify its exclusion. identify the data requirements needed to quantify
the baseline value using the chosen method.
Defining a range of baseline scenario options When selecting the baseline scenario method,
If possible, users should identify the single baseline consideration should be given to the data needs and
scenario that is considered most likely for each data availability under both the baseline scenario
impact being assessed. In certain cases, multiple and the policy scenario, since the same method or
baseline options may seem equally likely. In such model should be used for both scenarios.
cases, users should consider estimating and reporting
a range of results based on these alternative Multiple types of data can be used to estimate the
baseline scenarios. Users should conduct sensitivity impacts of policies, including bottom-up and top-
analysis to see how the results vary depending on down data (see Table 8.4).
the selection of baseline options. Sensitivity analysis
involves varying the parameters, or combinations Bottom-up and top-down data may be appropriate
of parameters, to understand the sensitivity of the in different contexts and are valuable for different
overall results to changes in those parameters. It is purposes. For example, top-down data may be
a useful tool for understanding differences resulting most appropriate for national policies, whereas
from methodological choices and assumptions, and bottom-up data may be better suited to smaller-scale
exploring model sensitivities to inputs. Sensitivity policies. The choice of bottom-up versus top-down
analysis is further described in Chapter 11. approaches depends on data availability and the
needs of the assessment.
Use of assumptions and expert judgment
Assumptions or expert judgment will likely be A wide range of tools and models can be used
required where information is not available to to quantify social, environmental and economic

TABLE 8.4

Overview of bottom-up and top-down data

Type of data Description

Bottom-up Bottom-up data are measured, monitored or collected at the facility, entity or project level. Examples
are energy used at a facility (e.g. using a measuring device such as a fuel meter) and production
output.

Top-down Top-down data are macro-level data or statistics collected at the jurisdiction or sector level. Examples
are national energy use, population, GDP and fuel prices. In some cases, top-down data are
aggregated from bottom-up data sources.

Source: Adapted from WRI (2014).

106 Sustainable Development Methodology

impacts. Methods range from simple equations example, some users may choose simple scenarios
(e.g. simple extrapolation) to complex models to support their analyses, whereas others may
(e.g. simulation models, computable general want to use additional variables, longer time scales
equilibrium models, integrated assessment models). or more detailed time steps, or have the flexibility
Simple equations may not be sufficient to represent to incorporate changing policies or patterns and
the complexity needed to accurately estimate develop conditional futures. Likewise, some may be
baseline or policy scenarios, or to capture the interested in assessing a small geographic region,
difference between them. Detailed models may be a single sector or even a single project, whereas
needed to estimate the impacts of certain policies. others may want multi-scale futures or integrated
Detailed models may also be appropriate when the approaches.25
chosen impact category includes multiple interacting
parameters. A suite of models may be available, with the choice
between models depending on users’ specific needs.
A variety of methods can be used, depending Models will require varying levels of data input, user
on what type of data is available and the level of knowledge and expertise, and cost. Selecting the
accuracy desired. Some methods (e.g. engineering most appropriate tool will depend on users’ available
models) calculate or model the impact of a policy for time and financial resources, as well as their team
each facility, project or entity affected by the policy, expertise. These considerations are illustrated in
then aggregate across all facilities, projects or entities Table 8.5.
to determine the total impact of the policy. Other
methods may include regression analysis or other Table 8.6 provides an overview of types of economic
statistical methods, simulation models, computable models for quantifying economic impacts. Box 8.5
general equilibrium models or other models. provides an explanation of one model for quantifying
job and economic impacts of constructing and
For example, a user assessing the impact of a solar operating power plants, such as wind farms. Box 8.6
PV incentive policy on jobs could use a bottom-up provides an example of a model for estimating the
approach by multiplying the estimated number health and economic effects of air pollution.
of buildings that install solar PV systems by the
estimated number of workers needed to install and
maintain solar PV systems per building, using data
provided by individual companies. Alternatively,
a user could use a top-down approach by using
economic models based on national employment
statistics on the number of people employed in the
solar energy industry and other relevant variables.
Hybrid approaches that combine elements of both
bottom-up and top-down approaches may also be
used.

The ICAT website24 provides examples of tools

and models to support impact quantification.
Users can use existing methods or models, or
develop new ones (if no relevant and appropriate
methods or models exist). Users should select a
tool that achieves sufficiently accurate results in the
context of objectives, data availability and resource
constraints. Objectives may range from theoretical
explorations of policy questions, to practical
applications of the results in a governmental
regulatory or programmatic context, to forecasting
for planning purposes. These needs will determine
the range of sectors that must be included in the
tool, the geographic scales and time frames. For

24
https://climateactiontransparency.org/icat-toolbox/sustainable-
development 25
USGCRP (2016).
 Part IV: Quantitative approach to impact assessment 107

TABLE 8.5

Considerations for selecting tools to assess social, economic or environmental impacts

Level of
depth/
accuracya Model capabilities Cost Ease of use Data inputs

Assumptions embedded in Up to Highly complex; use Highly data-

Higher the model are dynamic; can tens of requires trained experts, intensive; may rely
optimize for a specific variable thousands and significant time to on software of
or output; may produce a of dollars gather input data and models for inputs
range of quantitative outputs produce model output
(several weeks or months)

Assumptions embedded in No cost or Designed for use by the Not data-

the model are static; cannot low cost public: easy to navigate intensive; relies
optimize for a specific variable and run; requires limited on pre-populated
or output; may produce time to run (several hours data and default
Lower limited quantitative outputs or days) assumptions

a
The level of accuracy varies with the various attributes presented here. In reality, a complex, advanced model that has a high cost
and requires extensive data inputs will only be as accurate as the quality of the data that go into it.

TABLE 8.6

Overview of modelling approaches and tools for economic analysis

Method Advantages Disadvantages

Input– • Quantifies the total economic effects of a • Static; multipliers represent only a snapshot of
output change in the demand for a given product or the economy at a given point in time
model service • Generally assumes fixed prices
(also called • Can be inexpensive • Typically does not account for substitution
multiplier
effects, supply constraints, and changes in
analysis)
competitiveness or other demographic factors

Econometric • Usually dynamic; can estimate and track • Historical patterns may not be best indicator
models changes in policy impacts over time or predictor of future relationships
• Coefficients are based on historical data and • Some econometric models do not allow
relationships, and statistical methods can be foresight
used to assess model credibility

Computable • Accounts for substitution effects, supply • Not available for all regions
general constraints and price adjustments
equilibrium
models

Hybrid • Most sophisticated, combining aspects of all • Can be expensive

models the above
• Dynamic; can be used to analyse both short-
and long-term impacts
• Can be used to model regional interactions

Source: U.S. EPA (no date, a).

108 Sustainable Development Methodology

BOX 8.5

JEDI model for estimating job and economic impacts from power plants

The National Renewable Energy Laboratory’s Jobs and Economic Development Impact (JEDI) model is an Excel-based model
that estimates the number of jobs and economic impacts from constructing and operating power plants, fuel production
facilities and other projects at the local level. For example, JEDI estimates the number of construction jobs from a new wind
farm. JEDI models are used by decision makers, public utility commissions, potential project owners, developers and others.
The model estimates the project costs and the economic impacts in terms of jobs, earnings (i.e. wages and salaries) and
output (i.e. value of production) resulting from the project. Jobs, earnings and output are distributed across three categories:
project development and on-site labour impacts, local revenue and supply chain impacts, and induced impacts. The results
are more likely to better reflect the actual impacts from the specific project if the user can incorporate project-specific data
and the share of spending expected to occur locally. Project-specific data include a bill of goods (costs associated with actual
construction of the facility, roads, etc., as well as equipment costs, other services and fees required), annual operating and
maintenance costs, the portion of expenditures to be spent locally, financing terms and local tax rates. The analysis is not
designed to provide a precise forecast, but rather an estimate of overall economic impacts from specific scenarios.
The JEDI model uses an input–output methodology. It uses economic data (multipliers and consumption patterns) to
estimate the local economic activity and the resulting impact from new energy generation plants. This involves aggregating
national and regional economic and demographic data to calculate inter-industry linkages, the relationships between
changes in demand for goods and services, and the associated economic activity at the local and regional levels. Local
spending results from using local labour (e.g. concrete pouring), services (e.g. engineering, design, legal), materials (e.g. wind
turbine blades) or other components (e.g. nuts and bolts).

Source: NREL (no date).

BOX 8.6

The Benefits Mapping and Analysis Program (BenMAP) model for estimating the health
and economic effects of air pollution

The United States Environmental Protection Agency’s BenMAP-Community Edition (CE) tool estimates the economic value
of health impacts resulting from changes in air quality – specifically, ground-level ozone and fine particles. BenMAP-CE is
an open-source computer program that calculates the number and economic value of air pollution–related deaths and
illnesses. The software incorporates a database that includes many of the concentration–response relationships, population
files, and health and economic data needed to quantify these impacts.
Air pollution affects health through fine particles that penetrate deep into the lungs and enter the bloodstream. Health
impacts from particles include premature death, non-fatal heart attacks and aggravated asthma. Ground-level ozone is an
oxidant that can irritate airways in the lungs. Health impacts from ozone include premature death, aggravated asthma and
lost days of school.

A ”pyramid of effects“ from air pollution

The pyramid describes how the incidence and severity of fine
particle- and ozone-related health impacts are related. Health
Thousands
outcomes towards the bottom of the pyramid, such as asthma
attacks and cardiac effects, are less severe, and affect a larger
Magnitude of impacts

Severity of effects

Tens of proportion of the population. Impacts towards the tip of the

thousands
pyramid, such as hospital admissions and heart attacks, are more
severe and affect a smaller proportion of the population.
BenMAP-CE quantifies the impacts shown in white.
Millions
 Part IV: Quantitative approach to impact assessment 109

BOX 8.6, continued

The Benefits Mapping and Analysis Program (BenMAP) model for estimating the health
and economic effects of air pollution

BenMAP-CE calculates the economic value of air quality change using both “cost of illness” and “willingness to pay”
metrics. The cost of illness metric summarizes the expenses that an individual must bear for air pollution–related hospital
admissions, visits to the emergency
department and other outcomes; this metric
includes the value of medical expenses and
lost work, but not the value that individuals
place on pain and suffering associated
with the event. In contrast, willingness to
pay metrics account for the direct costs
noted above as well as the value that
individuals place on pain and suffering, loss
of satisfaction and loss of leisure time. This
simple example summarizes the procedure
for calculating economic values using these
two metrics in BenMAP-CE.

Source: U.S. EPA (no date, b).

8.3.4 Collect data for each indicator each indicator being estimated over defined time
periods, such as annually over the assessment
The next step is to collect data for each indicator (and period, if feasible. It is important to report the
parameter, if applicable) in each baseline scenario. methods, assumptions and data sources used. Users
To estimate baseline values for each indicator, users should also justify the choice of whether to estimate
should first decide whether to estimate new baseline new baseline values and assumptions or to use
values or use baseline values from published data published baseline values and assumptions. If no
sources. For some indicators, published values may data source is cited, users should provide sufficient
not be available. In this case, users should estimate information to enable stakeholders and others
new values. tracking the impact over time to know where to look
for updates to the data.
Users should collect data separately for different
groups in society, where relevant, such as men When collecting data from various data sources,
and women, people of different income groups, users should consider whether the data source
people of different racial or ethnic groups, people is readily available, whether data sources will be
of different education levels, people from different available to track indicator values over time, and
geographic regions, and people in urban versus rural how expensive or labour-intensive it will be to collect
locations. data over time. Users should use conservative
assumptions to define baseline values when
Either using published values or estimating new uncertainty is high or a range of possible values exist.
values, users should report the baseline values for Conservative values and assumptions are more likely
110 Sustainable Development Methodology

to overestimate negative impacts or underestimate Users should use high-quality, up-to-date and peer-
positive impacts resulting from a policy. reviewed data from recognized, publicly available,
credible sources, if available. When selecting
Parameters whose values will not change between data sources, users should apply the data quality
the baseline and policy scenario may “cancel out” indicators in Table 8.7 as a guide to obtaining the
when the baseline and policy values are subtracted. highest-quality data available. Users should select
Where that is the case, the value chosen for the data that are the most representative in terms of
parameter will not influence the final result, and technologies, practices, time and geography; the
fewer resources should be expended to gather most complete; and the most reliable.
the data for the parameter. Ideally, where such
parameters will cancel out in the final comparison, In some cases, the baseline scenario itself may be
the method should be simplified, and its description the subject of published research and available
narrowed to remove parameters that are not for use. As above, the information should be high
relevant. quality and credible. In addition, the method used
should be sufficiently clear that users can generate
Option 1: Using baseline values from published a comparable policy scenario, with consistent
data sources methods, assumptions and data sources.
In some cases, existing data sources of sufficient
quality may be available to determine baseline values For published values, a range of data may be
for indicators. Potential data sources of historical or available, such as:
projected data include published studies of similar
policies and impact categories in the same or other • international default values
jurisdictions, peer-reviewed scientific literature,
government statistics, reports published by • national average values
international institutions (such as the International
Energy Agency, IPCC, the World Bank, and the Food • jurisdiction- or activity-specific data.
and Agriculture Organization of the United Nations
– FAO), and economic and engineering analyses and In general, users should use the most accurate and
models. representative data available.

TABLE 8.7

Data quality indicators

Indicator Description

Technological The degree to which the data set reflects the relevant technologies, processes or
representativeness practices

Temporal representativeness The degree to which the data set reflects the relevant time period

Geographical The degree to which the data set reflects the relevant geographic location (e.g. country,
representativeness city, site)

Completeness The degree to which the data are statistically representative of the relevant activity.
Completeness includes the percentage of locations for which data are available and
used out of the total number that relate to a specific activity. Completeness also
addresses seasonal and other normal fluctuations in data.

Reliability The degree to which the sources, data-collection methods and verification procedures
used to obtain the data are dependable. Data should represent the most likely value of
the parameter over the assessment period.

Source: WRI (2014), based on Weidema and Wesnaes (1996).

 Part IV: Quantitative approach to impact assessment 111

Option 2: Estimating new baseline values

In some cases, no published baseline data and
assumptions will be available for historical
or projected data, or the existing data may
be incomplete, of poor quality, or in need of
supplementation or further disaggregation. Users
should estimate new baseline values when no
relevant data are available that support the level of
accuracy needed to meet the stated objectives.

To estimate new baseline values for a given indicator,

users should:

1. collect historical data for the indicator

2. identify other policies and non-policy drivers

that affect each indicator over the assessment
period, and make assumptions for those
drivers

3. estimate baseline values for each indicator,

based on historical data and assumptions
about drivers.

8.3.5 Estimate baseline values

for each indicator

The final step in developing the baseline is to apply

the method to the data collected to estimate baseline
values for each indicator.

It is a key recommendation to estimate baseline

values over the assessment period for each indicator
included in the assessment boundary. Any impact
included in the assessment boundary that cannot
be estimated should be assessed qualitatively (as
described in Chapter 7). It is a key recommendation
to separately estimate baseline values for different
groups in society, where relevant.

See Appendix A for an example of estimating the

impact of a solar PV incentive policy, including
estimating the baseline. The ICAT website26 provides
examples of tools and models to support impact
quantification.

26
https://climateactiontransparency.org/icat-toolbox/sustainable-
development
9 Estimating impacts ex-ante

This chapter describes how to estimate the expected 9.1 Define and describe the policy
future impacts of a policy (ex-ante assessment). In this scenario for each indicator
chapter, users estimate policy scenario values for the
indicators included in the assessment boundary. The In Chapter 8, users defined an indicator for each
impacts of the policy are estimated by subtracting impact category included in the assessment
baseline values (as determined in Chapter 8) from policy boundary. For examples of indicators, see Table 5.5.
scenario values (as determined in this chapter). This The indicators will be estimated for the baseline
chapter is structured around the steps in the scenario and policy scenarios to estimate the impact of
method, but the guidance is also helpful when using the the policy. Each indicator will generally require a
deemed estimates method (defined in Chapter 8). Users different assessment method. The same general
who are not quantitatively assessing impacts ex-ante assessment method(s) used to estimate the baseline
can skip this chapter. value (in Chapter 8) should be used to estimate the
policy scenario value for each indicator to ensure
Checklist of key recommendations methodological consistency between the baseline
and policy scenario estimations. Consistency ensures
• Define a policy scenario that represents the that the estimated impact reflects underlying
conditions most likely to occur in the differences between the two scenarios, rather
presence of the policy over time for each than differences in methods. If it is not feasible or
indicator being estimated, taking into appropriate to use the same method, users should
account all specific impacts included in the justify why different methods have been used.
quantitative assessment boundary The ICAT website27 provides examples of tools and
• Estimate the net impact of the policy on each models to support impact quantification.
indicator by subtracting baseline values from
policy scenario values, taking into account all It is a key recommendation to define a policy scenario
specific impacts included in the quantitative that represents the conditions most likely to occur
assessment boundary in the presence of the policy over time for each
• Separately assess the impacts of the policy indicator being estimated, taking into account
on different groups in society, where relevant all specific impacts included in the quantitative
assessment boundary. The policy scenario
represents the events or conditions most likely to

FIGURE 9.1

Overview of steps in the chapter

Define and describe the

Estimate policy scenario Estimate the net impact of
policy scenario for each
values for each indicator the policy on each indicator
indicator
(Section 9.2) (Section 9.3)
(Section 9.1)

27
https://climateactiontransparency.org/icat-toolbox/sustainable-
development
 Part IV: Quantitative approach to impact assessment 113

occur in the presence of the policy (or package of and the quantity of energy consumed in the baseline
policies) being assessed. The only difference between scenario and the policy scenario. In this example,
the baseline scenario and the policy scenario is that “household cost savings” is the indicator (measured
the policy scenario includes the changes caused by in dollars or other currency), and “electricity price”
the policy (or package of policies) being assessed. See and “quantity of energy consumed” are parameters.
Figure 9.2 for an illustration of estimating impacts ex- These two parameters are not themselves indicators
ante. Users can estimate policy scenario values either of interest, but are necessary to calculate the
before or after estimating baseline values. impact on the indicator of interest (“household cost
savings”). Calculating the impact on each indicator
Users should identify various policy scenario options therefore requires estimating policy scenario values
and choose the one considered to be the most likely for each parameter in the assessment method(s).
to occur in the presence of the policy. It is important
to consult stakeholders during the selection and To estimate policy scenario values for each
estimation of the policy scenario to ensure credibility. parameter, users should first identify which
Users should describe the policy scenario for each parameters are affected by the policy. In the example
indicator being estimated. above, “quantity of energy consumed” is affected
by the policy, since it is designed to save energy,
whereas “electricity price” is not affected by the
policy.
9.2 Estimate policy scenario values
for each indicator Parameters that are affected by the policy (such as
“quantity of energy consumed”) need to be estimated
For some indicators, it is possible to directly in the policy scenario. These parameter values are
estimate policy scenario values, without the need for expected to differ between the policy scenario and
additional parameters. Other assessment methods the baseline scenario. Users should estimate policy
require multiple parameters to estimate policy scenario values for these parameters by developing
scenario values for a given indicator. For example, assumptions about how the policy is expected to
estimating household cost savings from an energy affect each parameter over the assessment period
efficiency policy requires data on the electricity price (described further in Section 9.3). This follows the

FIGURE 9.2

Estimating impacts ex-ante

nario
ne sce
Baseli
INDICATOR VALUE

IMPACT
OF
POLICY
OR
Historical
ACTION
values
Policy
scenar
io

DATE OF ASSESSMENT TIME

Source: Adapted from WRI (2014).

114 Sustainable Development Methodology

same general process as estimating baseline values In general, users should use the most accurate data
in Section 8.3, but instead now is used to estimate available.
policy scenario values.
Option 2: Estimating new policy scenario
Parameters that are not affected by the policy (such values
as “electricity price”) do not need to be estimated In some cases, no relevant published data and
again, since the parameter value is not expected to assumptions will be available for policy scenario
differ between the policy scenario and the baseline values, or the existing data may be incomplete,
scenario. of poor quality, or in need of supplementation or
further disaggregation. Users should estimate new
Users should report the policy scenario values for policy scenario values and assumptions when no
each indicator being estimated, and the methods, relevant data are available that support the level of
assumptions and data sources used to calculate accuracy needed to meet the stated objectives.
policy scenario values.
Users can use a range of methods and data to
estimate policy scenario values, ranging from simpler
9.2.1 Guidance for estimating policy to more complex. For example, a simple method
scenario values may involve an assumption that parameters will
remain static (fixed) over the assessment period or
Users can either: involve a linear extrapolation of historical trends. A
more complex approach may involve an assumption
• use policy scenario values from published that parameters are dynamic (changing) over the
data sources (option 1), or assessment period; the values may be estimated
using detailed modelling or equations.
• estimate new policy scenario values (option 2).
Users should estimate the change in the indicator
Option 1: Using policy scenario values from over time, based on what is considered to be the
published data sources most likely scenario for each indicator. The most
In some cases, existing data sources of sufficient likely scenario can be based on evidence, such as
quality may be available to determine policy scenario peer-reviewed literature, modelling or simulation
values. Potential data sources of historical or exercises, government statistics, or expert judgment.
projected data include published studies of similar If scenarios or methods in existing literature are
policies and impact categories in the same or other not similar enough to use directly, users may need
jurisdictions, peer-reviewed scientific literature, to make adjustments to adapt the results found in
government statistics, reports published by literature to the assumptions made in the baseline
international institutions (such as the International scenario and other elements of the assessment.
Energy Agency, IPCC, the World Bank, FAO), and Users may also need to apply new methods,
economic and engineering analyses and models. models and assumptions not previously used in the
baseline method to estimate the expected change
Users should use high-quality, up-to-date and peer- in each indicator as a result of the policy. However,
reviewed data from recognized, publicly available, new methods should not be used to estimate
credible sources, if available. When selecting total impacts of the policy, since the same general
data sources, users should apply the data quality methods used to estimate baseline values should be
indicators in Table 8.7 as a guide to obtaining the used to estimate policy scenario values, to ensure
highest-quality data available. Users should select consistency.
data that are the most representative in terms of
technologies, practices, time and geography, and the Each indicator may be assumed to be static or
most complete. dynamic over the assessment period. Dynamic
indicators can change at a linear or non-linear
For published values, a range of data may be rate. In many cases, dynamic models that allow for
available, such as: conditions to change throughout the assessment
period are expected to be the most accurate, so they
• international default values should be used where relevant and feasible.

• national average values

• jurisdiction- or activity-specific data.

 Part IV: Quantitative approach to impact assessment 115

To estimate policy scenario values for each indicator In general, users should follow the most accurate
affected by the policy, users should consider a variety approach that is feasible, and focus on achieving
of factors (described in more detail below), such as: higher levels of accuracy for the most significant
impact categories and specific impacts included in
• historical trends and expected values in the the assessment boundary.
baseline scenario
Historical trends and expected values in the
• timing of impacts baseline scenario
Historical data can inform the expected future values
• barriers to policy implementation or of each indicator, in both the baseline scenario and
effectiveness the policy scenario. Understanding the historical
values of the indicator as well as the expected values
• policy interactions in the baseline scenario is useful when estimating
policy scenario values.
• sensitivity of parameters to assumptions.
Timing of impacts
To the extent relevant, users should also consider: Changes in policy scenario values depend on the
timing of expected impacts. There may be a delay
• non-policy drivers included in the baseline between when the policy is implemented and when
scenario (see Chapter 8), which should be impacts begin to occur. Impacts may also occur
different between the baseline and policy before policy implementation begins because of
scenarios if they are affected by the policy early action taken in anticipation of the policy.

• learning curves (economic patterns that can Users should assume that a policy will operate
accelerate or slow new product development indefinitely unless an end date is explicitly embedded
and deployment) in the design of the policy, even if there is uncertainty
about whether it will eventually be discontinued.
• economies of scale If the policy is limited in duration, the assessment
period may include some impacts that occur during
• technology penetration or adoption rates the policy implementation period and some that
(the pace of adoption by targeted actors, occur after the policy implementation period.
which may be slow initially then accelerate as
products become more socially accepted). Users should also consider whether and how the
implementation of the policy is expected to change
Depending on the assessment, users may not need during the assessment period. Examples are tax
to consider each of these factors. In practice, users instruments where the tax rate increases over time,
may also be limited by: performance standards where the level of stringency
increases over time, or regulations with multiple
• the type of policy (which may require distinct phases.
consideration of certain factors but not
others) In addition to estimating and reporting the full
impacts of the policy over the assessment period,
• the assessment method – for example, users can separately estimate and report impacts
simplified approaches may be limited to linear over any other time periods that are relevant. For
approximations example, if the assessment period is 2020–2030,
users can separately estimate and report impacts
• data availability (which may limit the number over the periods 2020–2025, 2025–2030 and 2020–
of factors that can be considered) 2030.

• objectives of the assessment (which may Barriers to policy implementation,

require a more or less complete and accurate enforcement or effectiveness
assessment) The policy scenario values should represent the
values most likely to occur in the presence of the
• available resources to conduct the policy, which depend on assumptions relating
assessment. to policy implementation, enforcement and
effectiveness. Depending on what is considered most
likely in a particular context, users should either
116 Sustainable Development Methodology

(1) estimate the maximum impacts of the policy if indicator. It is a key recommendation to estimate
full implementation is most likely, or (2) discount the net impact of the policy on each indicator by
the maximum impacts based on expected subtracting baseline values from policy scenario
limitations in policy implementation, enforcement values, taking into account all specific impacts
or effectiveness that would prevent the policy from included in the quantitative assessment boundary
achieving its maximum potential. For example, a (see equation 9.1). This involves estimating each
policy may not achieve its full potential because of specific impact within an impact category, then
governance challenges, such as a lack of capacity, aggregating across all of the specific impacts to
interagency coordination, public participation or determine the net impact of the policy on each
accountability. Users should apply conservative impact category, where feasible.
assumptions if there is uncertainty about the extent
of policy implementation and effectiveness. To do this, users should follow these steps for each
indicator being estimated:
Policy interactions
The policy assessed may interact with other 1. Estimate baseline values relating to each
implemented or adopted policies included in the specific impact in the quantitative assessment
baseline scenario. To accurately estimate policy boundary (as described in Chapter 8).
scenario values and the impacts of the policy, users
should determine whether the policy being assessed 2. Estimate policy scenario values relating
interacts with any policies included in the baseline to each specific impact in the quantitative
scenario (in either reinforcing or overlapping ways). assessment boundary.
For example, a new municipal solar PV incentive
policy may overlap with an existing national 3. Subtract baseline values from policy scenario
renewable energy mandate and a local energy values to estimate the impact of the policy for
efficiency policy. Because both existing policies are each specific impact.
included in the baseline scenario, they reduce the
energy savings achieved through the new solar 4. Aggregate across all specific impacts to
policy. estimate the total net impact of the policy on
a given indicator, which represents the change
If interactions with policies included in the baseline in the impact category, where feasible.
scenario exist, users should estimate the magnitude
of the policy interactions when estimating policy 5. Repeat the process for each indicator in the
scenario values. This enables estimation of the assessment boundary.
incremental impact of the policy being assessed
relative to existing policies included in the baseline When aggregating across impacts, users should
scenario.28 address any possible overlaps or interactions
between impacts to avoid overestimation or
Sensitivity of indicator values to assumptions underestimation of the total net impact of the policy.
Users should use sensitivity analysis to understand
the range of possible values of key indicators and Users should calculate baseline values, policy
parameters, and determine which scenario is most scenario values and the net impact of the policy over
likely. Users should also understand the range defined time periods (e.g. annually) and cumulatively
of uncertainty associated with key indicators and over the quantitative assessment period.
parameters. For more information on assessing
uncertainty and sensitivity analysis, see Chapter 11. Equation 9.1: Estimating the impact of the policy
on a given indicator
For a specific impact: Estimated change due to the
policy = policy scenario value for the chosen indicator
9.3 Estimate the net impact – baseline value for the chosen indicator
of the policy on each indicator
Net impact of a policy on the chosen indicator =
After estimating policy scenario values, the last step ∑ estimated change for each specific impact included
is to estimate the net impact of the policy on each in the assessment boundary

Note: “Net” refers to the aggregation of all specific impacts

included in the assessment boundary, including both positive
28
An example of assessing policy interactions is available in Del Río and negative impacts.
et al. (2013).
 Part IV: Quantitative approach to impact assessment 117

It is a key recommendation to separately assess the air pollution, the equation will yield a positive value
impacts of the policy on different groups in society, if the policy increases air pollution and a negative
where relevant. Examples of different groups are value if the policy reduces air pollution. If a policy
men and women, people of different income groups, creates jobs, the equation will yield a positive value,
people of different racial or ethnic groups, people whereas, if a policy reduces jobs, the equation will
of different education levels, people from different yield a negative value. Users may interpret and
geographic regions, and people in urban versus communicate the result as either positive or negative
rural locations. This allows users to understand or an increase or decrease, depending on the impact
distributional impacts on different groups, and category and the context.
manage trade-offs in cases where policies have
positive impacts on some groups and negative If any impacts in the quantitative assessment
impacts on others. boundary have not been estimated, users should
document and justify the exclusion, and describe the
Equation 9.1 results in a neutral estimate of impact, impact qualitatively (as explained in Chapter 7).
which may either be an increase (positive value) or
a decrease (negative value). Policy scenario values See Appendix A for an example of estimating the
may be either higher or lower than baseline scenario impact of a solar PV incentive policy. Table 9.1
values, depending on the impact being estimated. summarizes the ex-ante quantification results for the
For example, if estimating the impact of a policy on

TABLE 9.1

Estimated impact of the solar PV incentive policy on all impact categories included
in the assessment

Estimated impact
Impact category Indicator quantified (cumulative impact, 2016–2025)

Climate change mitigation GHG emissions (MtCO2e) from the electricity grid Reduction of 307 MtCO2e

Air quality/health impacts PM2.5 emissions (t) from the electricity grid Reduction of 1,177,996 t PM2.5
of air pollution
PM10 emissions (t) from the electricity grid Reduction of 2,437,234 t PM10

SO2 emissions (t) from the electricity grid Reduction of 4,265,161 t SO2

NOx emissions (t) from the electricity grid Reduction of 4,062,057 t NOx

Number of premature deaths per year in India Reduction of 32,304 premature

resulting from air pollution from coal plants deaths

Energy Renewable energy installed capacity (MW) Increase of 40,000 MW of renewable

energy capacity

Access to clean, Increase in number of houses/buildings/facilities Increase of 5,741,889 houses/

affordable and reliable with access to clean energy buildings/facilities with access to
energy clean energy

Capacity, skills and Number of new skilled trainees and workers on the Increase of 40,060 new skilled
knowledge development ground trainees and workers

Jobs Change in jobs (number of jobs) Net increase of 821,102 jobs

Income Savings in annual electricity bill for households and Savings of $27,855 million
businesses ($)

Energy independence Reduction in coal imports (t) Reduction of 57,770,140 t of coal

118 Sustainable Development Methodology

solar PV incentive policy across all impact categories 9.3.1 Separate reporting based on likelihood
included in the assessment. and probability, if relevant

Users should estimate total in-jurisdiction impacts Each impact of the policy included in the assessment
(the net change that occurs within the implementing may have a different likelihood of occurrence. In
jurisdiction’s geopolitical boundary) separately from Chapter 7, users categorize potential impacts based
total out-of-jurisdiction impacts (the net change on whether they are very likely, likely, possible,
that occurs outside the jurisdiction’s geopolitical unlikely or very unlikely to occur. If unlikely or very
boundary) for each indicator, if relevant and feasible. unlikely effects are included in the assessment, users
should consider reporting these impacts separately
Users should separately estimate and report the from the results for very likely, likely and possible
change resulting from each specific impact included impacts. Users can also separately report impacts
in the assessment boundary, where relevant and by each likelihood category (e.g. very likely, likely,
feasible. Users can also separately report by type of possible) if relevant and feasible.
impact.
Where likelihood is difficult to estimate, users can
When uncertainty is high (e.g. because of uncertain report a range of values for a given impact, based
baseline assumptions), users should report the net on sensitivity analysis for key parameters (further
impact of the policy on a given indicator as a range described in Chapter 11). Users can additionally
of likely values, rather than as a single estimate. incorporate probability into the estimation of ex-ante
Chapter 11 provides guidance on uncertainty and policy scenario values by weighting each impact by
sensitivity analysis. its expected probability (e.g. 100%, 75%, 50%, 25%,
0%).

Box 9.1 gives an example of a quantitative ex-ante

assessment in South Africa.

BOX 9.1

Quantitative ex-ante impact assessment in South Africa

A landfill in Garden Route District Municipality in South Africa was recently closed because of capacity constraints, and will be
replaced by a new regional waste management and landfill facility. The new landfill will not accept organic waste materials.
To inform the municipality’s new organic waste management plan, the South Africa Low Emission Development (SA-LED)
programme supported the municipality in conducting an ex-ante assessment of the sustainable development impacts of
different organic waste management options. The assessment focused on different approaches to managing abattoir waste,
which is a major component of organic waste in the district. The findings are expected to inform broader organic waste
management policy in the region.
Defining the baseline and policy scenarios: The baseline scenario assumed that the new regional landfill would be built
without an abattoir waste management facility, and the abattoir waste would go to other regional landfills, or be discarded
at the community or household level. The policy scenario assumed that the new waste management facility includes an
abattoir waste management facility that uses anaerobic digestion. The study quantified the impact of building the facility with
an abattoir waste management facility compared with the baseline scenario.
 Part IV: Quantitative approach to impact assessment 119

BOX 9.1, continued

Quantitative ex-ante impact assessment in South Africa

Determining impact categories and indicators to assess: Table 9.2 provides examples of impact categories and
indicators that were assessed.

TABLE 9.2

Examples of assessed impact categories and indicators

Impact category Indicator

Climate change mitigation • Amount of CO2e avoided (t/year)

Economic development • Earnings gained from the project (ZAR/year)

• GDP gained from the project (ZAR/year)

Jobs • Number of short-term jobs created, disaggregated by direct (on-site) and indirect
(supply chain) jobs
• Number of long-term operations and maintenance (O&M) jobs created,
disaggregated by direct and indirect jobs

Water saving • Amount of water saved (t/year)

Waste generation • Change in amount of waste sent to landfill (t/year)

Women employment • Number of full-time, trained women employees

Youth employment • Number of full-time, trained employees under 35 years old

Land use • Years of landfill life saved (years)

Identifying and assessing specific impacts: Based on the included impact categories, the study identified specific
impacts of the abattoir waste management facility. Each specific impact was qualitatively assessed, including its likelihood
and magnitude, to determine whether it was significant. With the exception of water savings, all impacts in Table 9.2 were
found to be significant. Because of data limitations, impacts on women employment and youth employment were assessed
qualitatively rather than quantitatively.
120 Sustainable Development Methodology

BOX 9.1, continued

Quantitative ex-ante impact assessment in South Africa

To quantify the baseline scenario, policy scenario and net impacts, the assessment used recent studies, including a
municipal waste characterization study performed by SA-LED, and tools such as the International Jobs and Economic
Development Impacts (I-JEDI) tool and the United States Environmental Protection Agency’s Waste Reduction Model (WARM)
tool. The quantitative results are shown in Table 9.3.

TABLE 9.3

Selected quantitative results for the waste management policy

Indicator Change

Change in GHG emissions from diverting waste to anaerobic digester Reduction of

5,718 tCO2e/year

Change in earnings gained from diverting waste to biopower Increase of

2,284,016 ZAR/year

Change in GDP gained from diverting waste to biopower Increase of

3,907,917 ZAR/year

Number of direct one-time construction jobs created in a single year Increase of 31 jobs

Number of indirect one-time construction jobs created in a single year Increase of 22 jobs

Number of direct long-term O&M jobs created from diverting waste to biopower Increase of 1 job

Number of indirect long-term O&M jobs created from diverting waste to biopower Increase of 1 job

Change in tonnes of waste sent to landfill Reduction of 9,697 t/year

Change in lifespan of new regional landfill site Increase of 3 years

10 Estimating impacts ex-post

Ex-post assessment is the process of estimating Checklist of key recommendations

historical impacts of policies. It is a backward-looking
assessment of impacts achieved to date. In this chapter, • Recalculate baseline values (as described in
users estimate the impact of the policy by comparing Chapter 8) every time an ex-post assessment
observed policy scenario values of an indicator (based is undertaken
on monitored data) with ex-post baseline values • Estimate the net impact of the policy on each
(described in Chapter 8). Unlike ex-ante assessment, indicator in the quantitative assessment
which involves forecasted values, ex-post assessment boundary by subtracting baseline values from
involves monitored or observed values. The impact of policy scenario values, taking into account all
the policy (ex-post) is estimated by subtracting baseline specific impacts included in the quantitative
values from policy scenario values. Users who are not assessment boundary
quantitatively assessing impacts ex-post can skip this • Separately assess the impacts of the policy on
chapter. Sections 10.1–10.4 apply to users following the different groups in society, where relevant
scenario method, while Section 10.5 applies to users • For users following the comparison group
following the comparison group method. method, identify an equivalent comparison
group for each impact category in the
assessment boundary, and collect data from
the comparison group and the policy group
over the assessment period for each indicator
included in the assessment boundary

10.1 Update baseline values or

ex-ante assessment (if relevant)

Figure 10.2 illustrates ex-post estimation of impacts.

In contrast to ex-ante policy scenario values, which
are forecasted based on assumptions, ex-post
policy scenario values are based on data collected

FIGURE 10.1

Overview of steps in the chapter

Update
Choose Estimate Estimate net Use
baseline
assessment policy scenario impact of the comparison
or ex-ante
method for values for each policy on each group method,
assessment, if
each indicator indicator indicator if relevant
relevant
(Section 10.2) (Section 10.3) (Section 10.4) (Section 10.5)
(Section 10.1)
122 Sustainable Development Methodology

FIGURE 10.2

Estimating impacts ex-post

INDICATOR VALUE

Ex-post baseline scenario

IMPACT OF
Historical
POLICY OR
values
ACTION
Ex-po
st po (EX-POST)
licy
(obse scenario
rved)

START OF POLICY DATE OF

IMPLEMENTATION ASSESSMENT

Source: Adapted from WRI (2014).

during the time the policy was implemented. Users apply a different method to estimate policy scenario
carrying out an ex-post assessment may estimate values. Users should choose the method that yields
ex-post policy scenario values either before or after the most accurate results. If both an ex-ante and
estimating ex-post baseline values. an ex-post assessment are carried out for the same
policy at different times, each assessment will likely
It is a key recommendation to recalculate baseline yield different estimates of the impacts of the policy,
values (as described in Chapter 8) every time an since the observed (ex-post) indicator values will
ex-post assessment is undertaken. The ex-post likely differ from assumptions forecasted in the ex-
baseline scenario should include all other policies ante scenario.
with significant impacts that were implemented
either before the implementation of the policy being
assessed or after the implementation of the policy
being assessed, but before the ex-post assessment. 10.2 Choose assessment method
for each indicator
The baseline scenario should also be recalculated
to include updates to all non-policy drivers, based This section provides a list of ex-post assessment
on their observed values over the assessment methods that can be used to estimate the impacts
period. Non-policy drivers should be considered in of a policy (see Table 10.1). The list is not exhaustive,
the baseline scenario if they are exogenous to the and users can classify methods differently depending
assessment – that is, if they are not affected by the on the individual context. Users can also use a
policy being assessed. combination of the approaches listed in Table 10.1.
The ICAT website29 provides specific examples of
If an ex-ante assessment for the policy was tools and models to support impact quantification.
previously carried out, the same method can be
used for the ex-post assessment, by replacing the
forecasted indicator values (ex-ante) with observed 29
https://climateactiontransparency.org/icat-toolbox/sustainable-
indicator values (ex-post). Alternatively, users can development
 Part IV: Quantitative approach to impact assessment 123

Users should select methods based on a combination the same method in a given situation, users should
of factors, such as data availability; the type of policy justify why different methods have been used.
and sector; the number of actors influenced by the
policy; the number of interacting policies; and the When selecting methods to estimate impacts ex-post,
capacity, resources and expertise available for each users should determine the desired level of accuracy
method. to be achieved. In general, users should follow the
most accurate approach that is feasible.
Users should ensure consistency in the methods
used to estimate baseline values and policy scenario
values for each indicator, to ensure that the
estimated impact reflects underlying differences
between the two scenarios, rather than differences
in method. If it is not feasible or appropriate to use

TABLE 10.1

Examples of ex-post assessment methods

Method Description

Collection of data Indicator values in the policy scenario are determined using data collected from affected
from affected participants, facilities or other actors. Data-collection methods may include monitoring of
participants, parameters (e.g. metering of energy consumption), collection of expenditure or billing data (e.g.
facilities or actors purchase records), or sampling methods.

Deemed The change in indicator values (rather than the policy scenario value of indicators) is estimated
estimates using previously estimated effects of similar policies. This involves collecting data on the number of
method actions taken (e.g. number of buildings that install rooftop solar PV) and applying default values for
the estimated impact or other relevant parameter per action taken (e.g. average reduction in grid-
connected electricity use per building that installs solar PV). The deemed estimate may be based
on published studies, equipment specifications, surveys or other methods. Deemed estimates are
used as a lower-cost method for policies that are homogeneous across policy contexts, such that
deemed estimates from other contexts are representative of the policy being assessed. Deemed
estimates can be complemented by sampling the affected participants or sources to determine
whether the estimates are sufficiently accurate and representative. In this approach, the impact is
estimated directly, without subtracting baseline values from policy scenario values. Baseline values
may be estimated as a subsequent step by adding or subtracting the deemed estimates from
observed policy scenario values.

Monitoring of Indicator values in the policy scenario are monitored using sector or subsector activity changes.
indicators In this case, the user may have limited or no information on end use or stock statistics, but may
have information on changes in relevant indicators for a sector (e.g. transportation, buildings) or
subsector (e.g. space heating in buildings). Policy scenario indicator values should be compared
with baseline indicator values to estimate the change.

Economic The change in indicator values (rather than the policy scenario value of indicators) is estimated
modelling by using econometric models, regression analysis, extended modelling such as input–output
analysis with price elasticities, or computable general equilibrium models. These types of models
are most appropriate for estimating economic impacts or estimating other types of impacts from
fiscal policies, such as taxes or subsidies. Economic models may specify that a dependent variable
(the indicator being assessed) is a function of various independent variables, such as the policy
being assessed, other policies and various non-policy drivers (e.g. prices, price elasticities of fuels,
economic activity, population). By doing so, models can control for various factors that affect the
impact category other than the policy being assessed.

Source: Adapted from WRI (2014).

124 Sustainable Development Methodology

10.3 Estimate policy scenario values 4. Aggregate across all specific impacts to
for each indicator estimate the total net impact of the policy on
a given indicator, which represents the change
Ex-post policy scenario values are based on data in the impact category, where feasible.
collected during the time the policy is implemented.
Users should first assess whether the specific 5. Repeat the process for each indicator in the
impacts identified in Chapter 6 actually occurred. assessment boundary.
This may include assessing the degree of policy
implementation to ensure that the policy was When aggregating across impacts, users should
implemented as planned, including assessing the address any possible overlaps or interactions
extent of enforcement and non-compliance, if between impacts to avoid overestimation or
relevant and feasible. underestimation of the total net impact of the policy.

Users should then update the impacts identified, Users should calculate baseline values, policy
based on observed data, before estimating each scenario values and the net impact of the policy
impact. To estimate certain impacts, users may over defined time periods, such as annually and
find it useful to conduct surveys with consumers cumulatively over the quantitative assessment
or businesses affected by the policy, or use results period.
from similar policy assessments, if the conditions are
similar enough for valid comparisons. Equation 10.1: Estimating the impact of the policy
on a given indicator
Users should report the policy scenario values for For a specific impact: Estimated change due to the
each indicator being estimated, and the methods, policy = policy scenario value for the chosen indicator
assumptions and data sources used to calculate – baseline value for the chosen indicator
policy scenario values.
Net impact of a policy on the chosen indicator = ∑
estimated change for each specific impact included
in the assessment boundary
10.4 Estimate net impact of policy
for each indicator “Net” refers to the aggregation of all specific impacts included in
the assessment boundary, including both positive and negative
impacts.
The last step is to estimate the net impact of the
policy. It is a key recommendation to estimate the net
impact of the policy on each indicator by subtracting It is a key recommendation to separately assess the
baseline values from policy scenario values, impacts of the policy on different groups in society,
taking into account all specific impacts included where relevant. Examples of different groups are men
in the quantitative assessment boundary (see and women, people of different income groups, people
equation 10.1). This involves estimating each specific of different racial or ethnic groups, people of different
impact within an impact category, then aggregating education levels, people from different geographic
across all the specific impacts to determine the net regions, and people in urban versus rural locations.
impact of the policy on each impact category, where This allows users to understand distributional impacts
feasible. on different groups, and manage trade-offs in cases
where policies have positive impacts on some groups
To do so, users should follow these steps for each and negative impacts on others.
indicator being estimated:
Equation 10.1 results in a neutral estimate of impact,
1. Estimate baseline values relating to each which may either be an increase (positive value) or
specific impact in the quantitative assessment a decrease (negative value). Policy scenario values
boundary (as described in Chapter 8). may be either higher or lower than baseline scenario
values, depending on the impact being estimated
2. Determine policy scenario values relating and the nature of the policy. Users may interpret and
to each specific impact in the quantitative communicate the result as either positive or negative
assessment boundary. or an increase or decrease, depending on the impact
category and the context.
3. Subtract baseline values from policy scenario
values to estimate the impact of the policy for If any impacts in the assessment boundary have not
each specific impact. been estimated, users should document and justify
 Part IV: Quantitative approach to impact assessment 125

the exclusion, and describe the impact qualitatively have been achieved (ex-post) compared with the ex-
(as described in Chapter 7). ante estimates.

See Appendix A for an example of estimating the

impact of a solar PV incentive policy.
10.5 Use the comparison group
Users should estimate total in-jurisdiction impacts method to estimate impacts
(the net change that occurs within the implementing (if relevant)
jurisdiction’s geopolitical boundary) separately from
total out-of-jurisdiction impacts (the net change This section provides guidance on using the
that occurs outside the jurisdiction’s geopolitical comparison group method to estimate the impacts
boundary) for each indicator, if relevant and feasible. of a policy.

Users should separately estimate and report the As outlined in Chapter 8, users can use the
change resulting from each specific impact included comparison group method to define the baseline
in the assessment boundary, where relevant and scenario when carrying out an ex-post assessment.
feasible. Users can also separately report by type of The comparison group method cannot be used for
impact. ex-ante assessments, since comparative data for the
comparison group and policy group during policy
When uncertainty is high (e.g. because of uncertain implementation cannot be obtained before policy
baseline assumptions), users should report the net implementation.
impact of the policy on a given indicator as a range
of likely values, rather than as a single estimate. The comparison group method involves comparing
Chapter 11 provides guidance on uncertainty and one group or region affected by a policy with an
sensitivity analysis. equivalent group or region that is not affected by
that policy. For users following the comparison
group method, it is a key recommendation to identify
10.4.1 Combining ex-ante and ex-post an equivalent comparison group for each impact
assessments category in the assessment boundary, and collect
data from the comparison group and the policy
Ex-ante and ex-post assessment may be combined in group over the assessment period for each indicator
a “rolling monitoring” approach. Under this approach, included in the assessment boundary. Any impacts
the forecast provided by the ex-ante assessment is in the assessment boundary that have not been
continually overwritten with the results from ex-post estimated should be documented and described
assessment, which allows comparison of the original qualitatively, with justification.
expectations and the final results. By combining
ex-ante and ex-post data, rolling monitoring can Figure 10.3 provides an overview of key steps.
demonstrate the impacts that have been initiated up
to a certain date (through ex-ante assessment), the
impacts that have been achieved up to a certain date
(through ex-post assessment), and the impacts that

FIGURE 10.3

Overview of steps for using the comparison group method

Identify the policy group and Collect data from the policy Estimate the impact of the
comparison group group and comparison group policy
126 Sustainable Development Methodology

10.5.1 Identify the policy group and see Box 10.1). If the groups are not sufficiently
comparison group equivalent, the comparison group method will
yield misleading results, so users should follow the
The first step is to identify the policy group (the scenario method instead (described in Chapter 8).
group or region affected by the policy) and the
comparison or control group (an equivalent group or
region not affected by the policy). The policy group 10.5.2 Collect data from the policy group
and comparison group may be groups of people, and comparison group
facilities, companies, jurisdictions, sectors or other
relevant groups. Users should collect data from both the policy
group and the comparison group for each indicator
Ideally, the policy group and the comparison group included in the assessment boundary. Users should
should be equivalent in all aspects except for the collect data from both groups at multiple points in
existence of the policy for the policy group and time to account for changes that occur over time.
absence of the policy for the comparison group. At a minimum, users should collect data from both
The most robust way to ensure that two groups are groups before and after the policy is implemented
equivalent is to implement a randomized experiment (in the policy group), so that the two groups can be
– for example, by randomly assigning one subset of compared during both the pre-policy period and the
entities to participate in a programme and the other policy implementation period.
subset to not participate in the programme.
Either top-down or bottom-up data (see
“Equivalent” means that the comparison group Section 8.3.3) may be used. To collect bottom-up
should be the same as, or similar to, the policy group data, representative sampling may be used to collect
in terms of:30 data from a large number of individual entities or
facilities. Appropriate statistical sampling procedures
• geography – for example, facilities in the should be used, and the sample size should be large
same city, subnational region or country enough to draw valid statistical conclusions.

• time – for example, facilities built within the

same time period 10.5.3 Estimate the impact of the policy

• technology – for example, facilities using the After data are collected, users should determine
same technology values without the policy (from the comparison
group) and values with the policy (from the policy
• other policies – for example, facilities subject group). In rare cases where the policy group and
to the same set of policies and regulations, comparison group are equivalent, the outcomes of
except for the policy being assessed each group can be compared directly. A statistical
test (such as a t-test) should be employed to ensure
• non-policy drivers – for example, facilities that the difference in values cannot be attributed
subject to the same external trends, such to chance. If the difference between the two groups
as the same changes in economic activity, is statistically significant, the difference can be
population and energy prices. attributed to the existence of the policy, rather than
to other factors.
When identifying a potential comparison group,
users should collect data from both the policy group In most cases, differences are expected to exist
and the comparison group before the policy is between the groups. If material differences exist that
implemented to determine whether the groups are may affect the outcome, users should use statistical
equivalent. Users should ensure that the entities in methods to control for variables other than the
the comparison group are not directly or indirectly policy that differ between the non-equivalent groups.
affected by the policy. Such methods are intended to address selection bias
and isolate the impact of the policy being assessed.
If the groups are similar but not equivalent, statistical See Box 10.1 for examples of methods that may be
methods can be used to control for certain factors used.
that differ between the groups (for examples,

30
Adapted from WRI (2014).
 Part IV: Quantitative approach to impact assessment 127

BOX 10.1

Examples of statistical methods for estimating impacts and controlling for factors
that differ between groups

Multiple regression analysis involves including data for each relevant driver that may differ between the groups
(e.g. economic activity, population, energy prices) as explanatory variables in a regression model, as well as proxies for other
relevant policies (other than the policy being assessed) that may differ between the two groups. If the expanded regression
model shows a statistically significant effect of the policy being assessed, the policy can be assumed to have an effect on the
policy group, relative to the comparison group. Statistical significance refers to the certainty that the difference between two
outcomes is unlikely to be a result of random chance.
Difference-in-difference methods compare two groups over two periods of time: a first period when neither the policy
group nor the comparison group implements a given policy, and a second period when the policy group implements
the policy and the comparison group does not. This method estimates the difference between the groups before policy
implementation (A1 – B1 = X), the difference between the two groups after policy implementation (A2 – B2 = Y), and the
difference between the two differences (Y – X) as a measure of the change attributable to the policy.
Matching methods are statistical approaches for making two groups (a policy group and a comparison group) more
equivalent, when random assignment is not possible.

Source: Adapted from WRI (2014).

11 Assessing uncertainty

This chapter provides an overview of concepts and assessing and managing uncertainty are most
procedures for understanding and evaluating the effective when done during, rather than after, the
uncertainty of the assessment. Uncertainty can be assessment process.
assessed either qualitatively or quantitatively. This
chapter is relevant to both qualitative and quantitative Sensitivity analysis is a useful method to test the
assessment of impacts. robustness of the assessment results. It involves
varying the value of key parameters (or combinations
Checklist of key recommendations of parameters) to determine the impact of such
variations on the overall results. Key parameters are
• Assess the uncertainty of the assessment those that are highly variable, highly uncertain or
results, either qualitatively or quantitatively most likely to significantly affect assessment results.
• For quantitative assessments, conduct a Sensitivity analysis can be conducted in combination
sensitivity analysis for key parameters and with uncertainty analysis to prioritize efforts for
assumptions in the assessment improving data. If a parameter is determined to
be highly uncertain and sensitive, users should
prioritize collecting better data for that parameter.
If a parameter is certain and insensitive, there is
11.1 Introduction to uncertainty less need for improving data quality. Figure 11.2
analysis and sensitivity analysis illustrates how to prioritize data improvement based
on uncertainty and sensitivity.
Understanding uncertainty is important for properly
interpreting and communicating the results of Understanding uncertainty can help users
the assessment. Uncertainty analysis refers to a understand whether to apply conservative
systematic procedure to quantify and/or qualify the assumptions. As explained in Chapter 3, accuracy
uncertainty associated with the impact assessment should be pursued as far as possible, but, once
results. Identifying, documenting and assessing uncertainty cannot be reduced to an acceptable level,
uncertainty can help users and stakeholders conservative estimates should be used.
understand the level of confidence they can have in
the results and identify the areas of the assessment
that contribute most to uncertainty. Users
should identify and track key uncertainty sources
throughout the assessment process. Identifying,

FIGURE 11.1

Overview of steps in the chapter

Review introduction
Communicate
(Section 11.1) Undertake Undertake sensitivity
uncertainty and
and types of uncertainty analysis analysis
sensitivity
uncertainty (Section 11.3) (Section 11.4)
(Section 11.5)
(Section 11.2)
 Part IV: Quantitative approach to impact assessment 129

FIGURE 11.2

Identifying where data improvement is needed in relation to uncertainty and sensitivity

DATA IMPROVEMENT
STRONGLY NEEDED

HIGH UNCERTAINTY HIGH UNCERTAINTY

LOW SENSITIVITY HIGH SENSITIVITY
UNCERTAINTY

LOW UNCERTAINTY LOW UNCERTAINTY

LOW SENSITIVITY HIGH SENSITIVITY

SENSITIVITY DATA IMPROVEMENT

LESS NEEDED

11.2 Types of uncertainty If parameter uncertainty can be determined, it can

typically be represented as a probability distribution
This chapter classifies uncertainty into three of possible values that include the chosen value
categories according to the source of uncertainty: used in the assessment. Individual parameter
parameter uncertainty, scenario uncertainty and uncertainties can be propagated to provide a
model uncertainty. The categories are not mutually quantitative measure of the uncertainty of the
exclusive, but they can be evaluated and reported in assessment results, which may be represented in the
different ways. Table 11.1 summarizes each type of form of a probability distribution.
uncertainty.

11.2.2 Scenario uncertainty

11.2.1 Parameter uncertainty
Ex-ante assessments involve baseline scenarios
Parameter uncertainty represents the imperfect and policy scenarios that describe how conditions
knowledge of true parameter values in an are expected to develop in the future, while ex-post
assessment method or model. It may arise from assessments involve baseline scenarios that describe
insufficient data, measurement errors, inaccurate how conditions would have developed in the past
approximation, or geographical and temporal if a policy were not implemented. These scenarios
variability. For example, wind speed may be used are based on a set of uncertain assumptions, which
as an input parameter to model the dispersion creates scenario uncertainty. To identify the influence
and concentration of PM2.5. The test equipment of these assumptions on the results, users should
will deliver wind speeds with a certain uncertainty. undertake a sensitivity analysis for key parameters in
Meanwhile, wind speed may vary every second, but the assumptions (described in Section 11.4).
only limited numbers of values (e.g. one value per
hour) will be used to model the dispersion of PM2.5.
130 Sustainable Development Methodology

TABLE 11.1

Types of uncertainty

Type of uncertainty Description

Parameter Uncertainty regarding whether a parameter value used in the assessment accurately
represents the true value of the parameter

Scenario Uncertainty of the calculated result due to various assumptions made in the baseline and
policy scenarios

Model Imperfect representation of modelling approaches, equations or algorithms to reflect the

real world

Source: Adapted from WRI (2014).

11.2.3 Model uncertainty Users should select an approach based on the

objectives of the assessment, the level of accuracy
Simplifying the real world into a numerical model needed to meet stated objectives, data availability,
introduces inaccuracies, and different models are and capacity and resources. Depending on the
likely to yield different results. For example, various methods used and data availability, users may not
life cycle impact assessment models can be used to be able to assess the uncertainty of all parameters
assess the environmental impacts associated with in the assessment method(s). Users should
producing solar PV panels. Each model is likely to assess the uncertainty of all parameters for which
yield different results, leading to model uncertainty. this assessment is feasible. Where quantitative
The extent of uncertainty can be estimated by uncertainty analysis is not possible or appropriate,
comparing the results of different models. Users uncertainty should be assessed and described
should acknowledge model uncertainties and report qualitatively.
model limitations qualitatively.

11.3.1 Qualitative uncertainty analysis31

11.3 Uncertainty analysis Qualitative uncertainty analysis can be done in a

variety of ways. This section outlines a structured
The two primary approaches to assessing uncertainty approach, which involves characterizing the level of
are: confidence of the results based on:

• qualitative uncertainty analysis • the quantity and quality of evidence (robust,

medium or limited)
• quantitative uncertainty analysis.
• the degree of agreement of the evidence
It is a key recommendation to assess the uncertainty (high, medium or low).
of the assessment results, either qualitatively or
quantitatively. Only qualitative uncertainty analysis is The level of confidence is a metric that can be
relevant to assessing the uncertainty of a qualitative expressed qualitatively to indicate certainty in
impact assessment. Either approach can be used the validity of a parameter value or result. (The
to assess the uncertainty of a quantitative impact qualitative confidence level described in this section
assessment. Quantitative uncertainty analysis can is distinct from statistical confidence and should not
provide more robust results than qualitative analysis. be interpreted in statistical terms.)
Reporting quantitative uncertainty estimates
also gives greater clarity and transparency to
stakeholders.

31
This section is adapted from IPCC (2010).
 Part IV: Quantitative approach to impact assessment 131

When characterizing parameter uncertainty, evidence low. As a rule of thumb, high agreement means
refers to the sources available for determining a that all sources had the same conclusion; medium
parameter value. Evidence should be assessed with agreement means that some sources had the same
regard to both its quantity and quality. Quantity conclusion; and low agreement means that most of
and quality of evidence can be classified as robust, the sources had different conclusions. This step is
medium or limited. Evidence should be considered not applicable if only one source is available.
robust when there is a large quantity of high-quality
evidence. Evidence should be considered medium A level of confidence provides a qualitative synthesis
when there is a medium quantity of medium- of the user’s judgment about the result, integrating
quality evidence. Evidence should be considered both the evaluation of evidence and the degree
limited when there is a small quantity of low-quality of agreement in one metric. Figure 11.3 depicts
evidence. High-quality evidence adheres to principles summary statements for evidence and agreement,
of research quality. Low-quality evidence shows and their relationship with confidence; confidence
deficiencies in adhering to principles of research increases as evidence and agreement increase. The
quality. Medium-quality evidence is a mix of high- level of confidence can be considered very high, high,
quality and low-quality evidence.32 medium, low or very low. In the best case (very high
confidence), the evidence found should be sourced
The degree of agreement of evidence is a measure from multiple credible, independent institutions.
of consensus or consistency across available sources Presentation of findings with “low” and “very low”
for a parameter value or result. The degree of confidence should be reserved for areas of major
agreement can be classified as high, medium or concern, and the reasons for their presentation

FIGURE 11.3

Summary statements for evidence and agreement, and their relationship with confidence

HIGH

High agreement High agreement High agreement

Limited evidence Medium evidence Robust evidence
CONFIDENCE SCALE
AGREEMENT

Medium agreement Medium agreement Medium agreement

Limited evidence Medium evidence Robust evidence

Low agreement Low agreement Low agreement

Limited evidence Medium evidence Robust evidence

LOW
EVIDENCE (TYPE, AMOUNT, QUALITY, CONSISTENCY)

Source: WRI (2014), adapted from IPCC (2010).

32
Adapted from DFID (2014).
132 Sustainable Development Methodology

should be explained. The confidence level of • Pedigree matrix approach from life cycle
individual parameters, models and scenarios should assessment (based on qualitative data quality
be aggregated to provide a level of confidence for the indicators in Table 8.7).
overall assessment, if feasible.
» This method provides a way to quantify
uncertainties based on a qualitative
11.3.2 Quantitative uncertainty analysis assessment of data. Five criteria are
provided in Table 8.7 to assess data
If feasible, users should carry out a quantitative quality from different perspectives. For
uncertainty analysis to characterize the uncertainty each criterion, a value is assigned by the
of key parameters. This involves estimating the practitioner to describe the data quality.
uncertainty of individual parameters (single These values can then be translated into
parameter uncertainty), then aggregating the the standard deviation of the data set.33
uncertainties for a given indicator as a whole
(propagated parameter uncertainty). Propagated • Survey of experts to generate upper- and
parameter uncertainty is the combined effect of each lower-bound estimates.
parameter’s uncertainty on the total result.
• The user’s expert judgment (based on as much
Users should estimate uncertainty at a specified data as available) or other approaches.
confidence level, preferably 95%. Users should
use the best available estimates from a variety of Once the uncertainties of individual parameters have
methods and approaches, such as a combination been estimated, they may be aggregated to provide
of measured data, published information, model uncertainty estimates for the entire assessment for
outputs and expert judgment. an indicator. Approaches to combining uncertainties
include:
Approaches to quantifying the uncertainty of
individual parameters include the following: • error propagation equations – an analytical
method used to combine the uncertainty
• Default uncertainty estimates for parameters associated with individual parameters from a
reported in literature. single scenario. Equations involve estimates
of the mean and standard deviation of each
• Probability distributions and standard input
deviations.
• Monte Carlo simulation – a form of random
» This method is feasible and preferred sampling used for uncertainty analysis that
when a large amount of data is available shows the range of likely results based on
for a given parameter. In such cases, the range of values for each parameter and
it is possible to generate a probability probabilities associated with each value.
distribution and other statistical values, To perform Monte Carlo simulation, input
such as standard deviations, which can be parameters must be specified with probability
propagated to the uncertainty of the final distributions. The input parameters are
output. varied at random but restricted by the given
probability distribution for each parameter.
• Uncertainty factors for parameters reported in Repeated calculations produce a probability
literature. distribution of the predicted output values,
reflecting the propagated uncertainty of
» One application of uncertainty factors is in the various parameters. This method gives
environmental assessments relating to risk comprehensive results, but is more resource-
and safety. For example, when assessing and time-intensive. Simple Monte Carlo
the toxicity impact of a certain chemical, simulations can be done using the Crystal Ball
experiments may be conducted on a small tool in Microsoft Excel.
group of people. To extrapolate the test
results to a larger group, an uncertainty
factor is applied to ensure maximum
protection and safety. This method is
especially relevant when conservative
methods are applied. 33
For more information, see Weidema and Wesnaes (1996).
 Part IV: Quantitative approach to impact assessment 133

Further references on quantitative uncertainty To conduct a sensitivity analysis, users should adjust
analysis the value of key parameters to determine the impact
For more detailed guidance on the methods outlined of such variations on the overall results. Since an
in this section, see the following references: assessment may include many impact categories
and involve many parameters, users should conduct
• IPCC (2000). Good Practice Guidance sensitivity analysis only on key parameters.
and Uncertainty Management in National
Greenhouse Gas Inventories34 Users should consider reasonable variations in
parameter values. Not all parameters need to be
• IPCC (2006). Chapter 3, Uncertainties. In 2006 subjected to both negative and positive variations
IPCC Guidelines for National Greenhouse Gas of the same magnitude, but they should be varied
Inventories, vol. 135 based on what is considered reasonable. Past trends
may be a guide to determining the reasonable range.
• World Resources Institute (WRI) and World As a general rule, variations in the sensitivity analysis
Business Council for Sustainable Development should at least cover a range of +10% and –10%
(WBCSD) (2003). Aggregating Statistical (unless this range is not deemed reasonable under
Parameter Uncertainty in GHG Inventories: the specific circumstances).
Calculation Worksheets36
Sensitivity analysis can be conducted in several
• WRI and WBCSD (2003). GHG Protocol ways. One simple method is to assess the relative
Guidance on Uncertainty Assessment in GHG sensitivity for one parameter at a time, according to
Inventories and Calculating Statistical Parameter equation 11.1.
Uncertainty37
Equation 11.1: Assessing the sensitivity of a
• WRI and WBCSD (2011). Quantitative Inventory parameter
Uncertainty38
∆output/output
• WRI and WBCSD (2011). Uncertainty Assessment S = ∆input
Template for Product GHG Inventories.39 /input
In the equation, S represents the relative sensitivity
11.4 Sensitivity analysis of the assessment output to the specific input
parameter. Input and output represent the original
A sensitivity analysis involves varying the value of values. ∆input is the marginal change in the input
key parameters (or combinations of parameters) parameter, which should represent a reasonable
to determine the impact of such variations on expected change. ∆output is the corresponding
the overall results. Sensitivity analysis is a useful marginal change in the output parameter. Using this
tool to understand differences resulting from equation, users can compare the sensitivity of the
methodological choices and assumptions, and to output in response to different input parameters.
explore model sensitivities to input parameters.
See Box 11.1 for an example of applying
For quantitative assessments, it is a key equation 11.1 to determine which of various
recommendation to conduct a sensitivity analysis for parameters is most sensitive.
key parameters and assumptions in the assessment.
Sensitivity analysis is expected to be most relevant
for quantitative impact assessments, but may also be
useful for certain qualitative impact assessments.

34
Available at: www.ipcc-nggip.iges.or.jp/public/gp/english.
35
Available at: www.ipcc-nggip.iges.or.jp/public/2006gl.
36
Available at: www.ghgprotocol.org.
37
Available at: www.ghgprotocol.org.
38
Available at: www.ghgprotocol.org.
39
Available at: www.ghgprotocol.org.
134 Sustainable Development Methodology

BOX 11.1

Example of sensitivity analysis

Table 11.2 illustrates a sensitivity analysis of three key parameters for a solar PV incentive policy. It is assumed that there
are 186,306,371 grid-connected households in India, with an annual consumption of 900 kilowatt-hours (kWh) electricity
per year per household. In the original policy scenario, 10% of existing grid-connected households are expected to adopt
rooftop solar PV systems and will be able to rely on solar for the entire household electricity demand. The other 90% of grid-
connected households will rely on a combination of grid-connected electricity and back-up diesel generators for electricity,
assuming that 90% (810 kWh) is supplied by the grid and 10% (90 kWh) is supplied by a diesel-fuelled power generator when
blackouts occur.
The three chosen parameters for sensitivity analysis are annual electricity consumption per household, the percentage of
households that will adopt solar PV, and the percentage of electricity supplied by grid for the households that use combined
electricity supply, assuming that the remaining electricity demand is met by diesel-fuelled power generators. Table 11.2
illustrates a scenario in which each parameter value is set to a reasonable assumption. The table also shows calculation of
the output – in this case, changes in emissions for each scenario. This example specifically focuses on PM10. Combined, this
information enables calculation of relative sensitivity. The input, output and sensitivity analysis results are presented below.

TABLE 11.2

Sensitivity analysis of estimated PM 10 emissions

Annual electricity Percentage of households Percentage of electricity

Parameter consumption that adopt solar PV supplied by grid

Input

Original value (kWh) 900 10% 90%

Scenario value (kWh) 1,800 80% 50%

Δinput/input 100% 700% –44%

Output: emissions reduction

Original value (t PM10) 300,817 300,817 300,817

Scenario value (t PM10) 601,635 71,886 171,695

Δoutput/output 100% –76% –43%

Sensitivity analysis result

Relative sensitivity 100% –11% 97%

This sensitivity results show that, of the three parameters, PM10 emissions are more sensitive to annual electricity
consumption and percentage of electricity supplied by the grid, and less sensitive to percentage of households that adopt
solar PV. This information can be used to prioritize future data-collection efforts.
 Part IV: Quantitative approach to impact assessment 135

11.5 Communicating uncertainty

and sensitivity

Reporting information about uncertainty helps

users and stakeholders assess the accuracy and
uncertainty of the reported results, to inform how
the information should be used. It is important to
properly communicate the results, since the estimate
of policy impact may not be very accurate, depending
on the methods, assumptions and data sources that
were used to assess the impacts.

Uncertainty can be reported in many ways, including

qualitative descriptions of uncertainty sources
and quantitative representations, such as error
bars, histograms and probability density functions.
Users should provide as complete a disclosure of
uncertainty information as possible.

Users should report a quantitative estimate or

qualitative description of the uncertainty of the
results. They should also report the range of results
from sensitivity analysis for key parameters and
assumptions.

Users should report the range of possible outcomes

based on different parameter values (representing
upper and lower bounds of plausible values) to
indicate the level of uncertainty. When uncertainty
is high, users should consider reporting a range
of values around the average or most likely value,
rather than only a single value. Users should
transparently report the full range of likely values,
rather than reporting only upper-bound or lower-
bound values.

Users should also use an appropriate number of

significant figures, depending on the uncertainty of
the results, to avoid overstating the precision of the
results.

Users should make a thorough yet practical effort

to communicate key sources of uncertainty in the
results, including key parameters and assumptions
that have high uncertainty. If feasible, users should
report both qualitative and quantitative uncertainty
information. They should also describe their efforts
to reduce uncertainty in future revisions of the
assessment, if applicable.
PART V
Monitoring and reporting
12 Monitoring performance over time

Monitoring helps users assess whether a policy is Checklist of key recommendations

on track and being implemented as planned. This
chapter provides guidance on how to (1) monitor the • Define indicators that will be used to track
performance of a policy over time by tracking the performance of the policy over time for each
progress of key indicators, (2) collect data needed for impact category included in the assessment
ex-post assessment and (3) prepare a monitoring plan. • If estimating impacts ex-post, collect data
needed for ex-post assessment
This chapter is relevant to users who want to: • Create a plan for monitoring indicators
• Monitor each of the indicators over time, in
• determine whether policies are being accordance with the monitoring plan
implemented as planned and having the desired • Separately monitor indicators for different
effects across the identified impact categories, groups in society, where relevant
to improve implementation and inform future
policy design

• assess progress towards achieving SDGs, to 12.1 Define approach to monitoring

adjust current efforts and inform future goal-
setting Monitoring during policy implementation serves two
distinct objectives:
• collect data needed for ex-post assessment of
impacts. • Monitor performance of the policy. Track key
indicators over time in relation to historical
values, goal values and values at the start of
policy implementation to understand whether
the policy is on track and being implemented
as planned.

• For ex-post assessment of impacts, collect

data on the indicators and parameters needed
(if applicable).

FIGURE 12.1

Overview of steps in the chapter

Collect
Define
parameters Define the Track
Define indicators Create a Monitor
needed to monitoring progress
approach to for monitoring indicators
calculate period and towards
monitoring monitoring plan over time
impacts ex- frequency SDGs
(Section progress (Section (Section
post (Section (Section
12.1) (Section 12.5) 12.6)
(Section 12.4) 12.7)
12.2)
12.3)
138 Sustainable Development Methodology

Users can collect data to meet one or both start of policy implementation. Each is described
objectives. The first objective requires the tracking below and illustrated in Figure 12.2:
of indicators only, while the second objective may
require tracking a broader set of parameters. • Relative to historical values. Monitor the
Indicators are metrics that can be monitored over trend in a given indicator over time to see
time to enable tracking of changes towards targeted whether it is moving in the right direction in
outcomes. Parameters are additional data needed relation to past values.
under certain circumstances to calculate the impact
of a policy on indicators that cannot be directly • Relative to goal values. Monitor the trend
monitored. in a given indicator in relation to goal values
(defined ex-ante) to see whether goals for that
Monitoring key indicators is useful for understanding indicator are being achieved.40
progress over time; understanding whether
indicators of interest are moving in the right • Relative to values at the start of policy
direction; and tracking progress towards meeting implementation. Monitor the trend in a
goals, such as sustainable development goals at the given indicator before and after a policy is
international, national or local levels. Monitoring key implemented to see whether the policy is
indicators over time is generally simpler and less having the desired effect.
onerous than estimating impacts, and can provide a
low-cost way of understanding policy effectiveness However, monitoring indicators is not sufficient to
by tracking trends in key indicators. If progress of key estimate the impact of a policy. Monitoring trends in
indicators is not on track in relation to goal values, indicators can show a correlation between desired
monitoring can inform corrective action. outcomes and the implementation of the policy, but
it does not demonstrate causation or allow changes
Key indicators can be monitored over time relative in indicators to be attributed to policies. Changes in
to historical values, goal values and/or values at the indicators could be a result of factors other than the
policy being assessed. Attributing impacts to specific

FIGURE 12.2

Monitoring indicators relative to historical values, goal values and the date of policy implementation

Goal value
INDICATOR VALUE

Historical
values

DATE OF POLICY TIME

IMPLEMENTATION

40
Tracking of indicators over time may still be useful even if there
are no defined goal values for the selected indicator.
 Part V: Monitoring and reporting 139

policies requires a baseline scenario, as discussed plan (see Section 12.5), and in relation to historical
in Chapters 8–10. Depending on how indicators are values and/or goal values, and to values at the start
defined, it may be possible to infer causation. For of policy implementation. The selected indicators
example, a user can monitor the number of new jobs from each impact category should be discussed in an
created from discrete projects resulting from a policy inclusive stakeholder consultation process to obtain
to demonstrate the additional jobs created. stakeholder perspectives and make the assessment
more complete. Chapter 8 of the ICAT Stakeholder
Users who are estimating the impacts of a policy Participation Guide provides more information on
ex-post should collect data on the broader range of how to conduct consultations.
parameters that are needed to calculate the ex-post
policy scenario and ex-post baseline scenario. The Users tracking progress towards SDGs may reference
types of parameters that need to be collected should the relevant SDG goal and, if applicable, the relevant
be informed by the ex-post estimation method that SDG target(s) for each selected indicator (as
will be used. To ensure an accurate assessment, data described in Section 12.7).
collection should begin before or at the beginning
of the policy implementation period and continue Table 12.1 provides an overview of possible impact
throughout the policy implementation period. categories and referenced SDGs, indicators, and brief
explanations of the indicators for a solar PV incentive
policy.

12.2 Define indicators for monitoring

progress of a policy
12.3 Collect parameters needed
It is a key recommendation to define indicators that to calculate impacts ex-post
will be used to track performance of the policy (if relevant)
over time for each impact category included in the
assessment (as defined in Chapter 5). For ex-post quantitative impact assessments, it is
necessary to identify and collect parameters needed
Examples of indicators are provided in Table 5.5. For to calculate impacts of the policy on each indicator
further guidance and examples of indicators that can being quantified. If estimating impacts ex-post, it is
be used, see: a key recommendation to collect data needed for ex-
post assessment. Parameters should be collected, as
• United Nations SDG website41 needed, for each impact category and each selected
indicator included in the assessment boundary (as
• United Nations SDG indicators website,42 described in Chapter 5).
including the global SDG indicators database43
and list of indicators44 Parameters are additional data needed under certain
circumstances to calculate the impact of a policy
• United Nations Indicators of Sustainable on indicators that cannot be directly monitored.
Development: Guidelines and Methodologies.45 For example, to estimate the impact category of
cost savings from a solar PV incentive policy that
When selecting indicators, users should consider replaces kerosene use (in the baseline scenario) with
the intended objectives of monitoring, the nature solar electricity, the indicator could be household
of the policy, the impact categories being assessed savings (money). Money saved is not monitored
and any related goals, stakeholder priorities, and directly. Instead, the parameters needed to calculate
data availability. All relevant indicators should be the amount of money saved include the cost of
clearly described. The selected indicators should kerosene and the amount of kerosene savings. These
be monitored in accordance with the monitoring parameters are needed to calculate the impact on
the selected indicator (money saved) but not the
indicator itself. Parameters can be collected from
various sources, such as statistics collected at the
41
https://sustainabledevelopment.un.org/sdgs
jurisdiction level or surveys.
42
http://unstats.un.org/sdgs
43
http://unstats.un.org/sdgs/indicators/database
44
http://unstats.un.org/sdgs/indicators/indicators-list
45
Available at: https://sustainabledevelopment.un.org/content/
documents/guidelines.pdf.
140 Sustainable Development Methodology

TABLE 12.1

Example of selected indicators and referenced SDGs for a solar PV incentive policy,
and explanations of chosen indicators

Impact category Indicator Explanation of chosen indicator

Energy (SDG 7) • Solar capacity installed (MW) These indicators will track the quantity of renewable energy
• Electricity delivered from solar installed and generated from the solar PV incentive policy.
PV installations (MWh)

Health (SDG 13) • Emissions of PM2.5, PM10, SO2 The policy will improve health of people by avoiding burning
and NOX of kerosene/paraffin, which causes severe indoor air pollution
• Number of premature deaths by emitting noxious fumes and soot. Kerosene lighting is
due to air pollution hazardous, and is responsible for many burns and deaths. The
policy will also improve health-care conditions by providing
• Number of health clinics
lighting and refrigeration for health clinics.
electrified

Quality of life • Number of households having The policy will provide more reliable lighting conditions,
(SDGs 1, 2, 16) access to clean, reliable and allowing children to study at home, which has a significant
affordable electricity impact on improving child education in rural families and
future employability. With a more reliable light source, adults
can pursue productive activities in the house after nightfall.

Access to clean • Share of people having access In the absence of reliable grid electricity, people depend
energy/energy to reliable electricity services mostly on diesel generators and kerosene/paraffin lamps
security (SDG 7) for lighting. The policy will make people less dependent on
expensive fuels and reduce the need to purchase fuel. The
policy will enable use of local energy sources, independent of
geopolitical uncertainty.

Empowerment of • Share of female The policy will create opportunities for new income-generating
women (SDG 5) entrepreneurs activities for women and women’s associations.

Employment/job • Number of people (men/ The policy will encourage new job-creating and income-
creation and women) in jobs generating activities related to renewable energy supply
income generation • Household income and installation, mini-grid operation, awareness raising, and
(SDG 8) marketing and accounting, thereby creating many new jobs.
The generation of income will enhance economic growth and
provide the means to afford electricity.

Economic • Number of households The policy will foster productivity, increase production
productivity (SDG 8) with improved economic efficiency and enable added-value activities.
productivity

Food security • Number of households with The policy will reduce food waste by improving refrigeration.
(SDG 2) improved food security It will also promote better food processing, adding value to
agricultural products.

Safety (SDG 3) • Number of people affected by Kerosene/paraffin lighting is hazardous and is responsible
hazardous conditions for loss of property through fire, as well as burns and death.
The policy will foster the implementation of safety measures
such as street lighting, security lighting, remote alarm systems,
electric fences and road signs.
 Part V: Monitoring and reporting 141

12.4 Define the monitoring • monitoring period

period and frequency
• monitoring frequency (fixed ex-ante during
Next, users should define the monitoring period and the monitoring period)
monitoring frequency.
• measurement or data-collection methods
(such as survey or census)
12.4.1 Monitoring period
• historical value (baseline value)
The monitoring period is the time period over which
the policy is monitored. At a minimum, the monitoring • goal value
period should include the policy implementation
period. Where possible, monitoring should also include • entity(ies) or institution(s) responsible for
pre-policy monitoring of relevant activities before monitoring the respective indicator and
implementation of the policy and post-policy monitoring collecting parameter(s), if applicable.
of relevant activities after the policy implementation
period. For example, a solar PV incentive policy that Additional information may include:
has a policy implementation period of 2015–2025 may
have a monitoring period of 2013–2027. Depending on • methods for generating, storing, collating and
the impact categories and indicators being monitored, reporting data
it may be necessary to monitor some indicators over
different time periods than others. In general, the • level of uncertainty of data and how this
longer the time series of data that is collected, the more uncertainty will be accounted for
robust the assessment will be.
• databases, tools or software systems to be
used for collecting and managing data
12.4.2 Monitoring frequency
• procedures for internal auditing, quality
Users can monitor indicators at various frequencies, assurance and quality control, including
such as monthly, quarterly or annually. In general, record keeping and internal documentation
users should collect data with as high a frequency procedures, and length of time data will be
as is feasible and appropriate in the context of archived
objectives. The appropriate frequency of monitoring
should be based on the needs of decision makers • whether data are verified and, if so,
and stakeholders, the type of impact categories verification procedures used
and indicators being monitored, cost, and data
availability. In general, the more frequently data are • roles and responsibilities of relevant
collected, the more robust the assessment will be. personnel involved in monitoring
The monitoring frequency should, in general, be fixed
ex-ante for the duration of the monitoring period. • competencies required and any training
needed to ensure that personnel have the
necessary skills.

12.5 Create a monitoring plan Before monitoring begins, users should identify
the entity or institution responsible for collecting
A monitoring plan is important to consistently data during the monitoring period. The responsible
track progress of indicators over time in relation to entity should establish a database based on the
goals. It is a key recommendation to create a plan for monitoring plan. See Box 12.1 for more information
monitoring indicators. on institutional arrangements for monitoring.

A monitoring plan should include the following key Table 12.2 provides an example of a template
elements: that can be used for a monitoring plan. The table
includes goal values and historical values for each
• brief description of each indicator previously identified indicator for a solar PV incentive
policy. Historical values were determined through
• source of data for each indicator and interviews with the communities that will benefit
parameter (if applicable) from the policy. Goal values should be estimated
142 Sustainable Development Methodology

BOX 12.1

Institutional arrangements for coordinated monitoring

Information on key performance indicators and parameters can be dispersed among different institutions. Given the wide
variety of data needed for impact assessment and the range of stakeholders involved, strong institutional arrangements
serve an important function. They play a central role in coordinating monitoring. A technical coordinator, or a coordinating
team or body is often assigned to lead monitoring, reporting and verification (MRV) processes in which responsibilities have
been delegated to different institutions. Since data can be widely dispersed between institutions, the coordinating body
oversees the procedures for data collection, management and reporting.
Countries may already have institutions in place as part of a national MRV system. In this case, users can consider expanding
the national MRV system to monitor the impact of the policy. Where strong institutional arrangements do not yet exist,
countries can determine the governmental body with adequate capacity and authority to be responsible for the MRV
system, and to establish the necessary legal arrangements. Institutional mandates help to strengthen the procedures and
the system, and may also help secure funding from the government to ensure the continuity of the process. Users can refer
to the UNFCCC Toolkit for non-Annex I Parties on Establishing and Maintaining Institutional Arrangements for Preparing
National Communications and Biennial Update Reports,45 as well as other sources, for support on establishing or improving
the institutional arrangements for a robust MRV system.

through inclusive consultations with a wide variety

of stakeholder groups, such as beneficiaries,
government representatives, technical experts,
businesses, NGOs and local representatives of
international organizations.

Box 12.2 presents an example of a monitoring plan

in South Africa.

If surveys are used and/or sampling procedures are

applied, users should develop a statistically sound
sampling plan as part of the monitoring plan. Users
should follow internationally recognized standards
for sampling. Before including the sampling plan
in the monitoring plan, users should familiarize
themselves with different standards and required
sampling sizes to ensure statistically sound results.

46
Available at: http://unfccc.int/files/national_reports/non-annex_i_
natcom/training_material/methodological_documents/application/
pdf/unfccc_mda-toolkit_131108_ly.pdf.
 Part V: Monitoring and reporting 143

TABLE 12.2

Example of a monitoring template for selected indicators and parameters for a solar PV
incentive policy

Responsible Historical Goal

Source of Monitoring Measurement entity or value in value
Indicator data frequency method institution 2015 for 2022

Rooftop solar Government Monthly Name plate Ministry of

capacity installed statistics showing installed Energy
capacity; ground
verification on a
random sample
basis

Electricity delivered Government Monthly Electricity Ministry of

from solar PV statistics meters; ground Energy
installations verification on a
random sample
basis

Number of health Survey Annual Community-level Health

clinics electrified assessment Ministry

Number of Survey Annual Community-level Ministry of

households having assessment Energy
access to clean
electricity

Number of people Survey Annual Community-level Ministry of

having access to assessment Energy
electricity services

Number of female Survey Annual Community-level Ministry of

entrepreneurs assessment Social Affairs

Number of Government Monthly Community-level Ministry of

people in jobs, statistics assessment Social Affairs
disaggregated by
gender

Money saved Statistics Biennial Sector-level Ministry of

through and/or assessment (cost Energy
replacement of survey of kerosene);
kerosene by solar community-level
energy (which assessment
requires further (amount of
parameters to kerosene saved)
calculate cost
of kerosene,
and amount of
kerosene saved
144 Sustainable Development Methodology

BOX 12.2

Defining indicators and a monitoring plan in South Africa

The draft White Paper on the Conservation and Sustainable Use of South Africa’s Biological Diversity is a strategy to
address biodiversity protection and sustainable utilization in South Africa. The white paper identifies six goals that cover
environmental, social and economic impacts. It lists 175 policy interventions to achieve these goals. The policy interventions
include controls on conservation and sustainable use of biodiversity, improving knowledge, direct biodiversity conservation
or rehabilitation activities, coordination and cooperation processes, relationship building and conflict resolution, capacity-
building, and monitoring. The Energy Research Centre at the University of Cape Town conducted an ex-ante qualitative
assessment of the strategy. As part of the assessment, the centre provided examples of indicators that can be tracked as
part of a monitoring plan (Table 12.3).

TABLE 12.3

Example of a monitoring template for a biodiversity policy in South Africa

Responsible
Source of Monitoring Measurement entity or Goal value for
Indicator data frequency method institution year Y

Areas protected Provincial National Land survey SANBI, with By 2028, in

(ha, km, km2) conservation Biodiversity support of protected areas:
authorities, Assessments DEA and CSIR 10.8 m land-
South Africa’s are updated based hectares,
Scientific every 7 years 353 km inshore;
Authority 210,000 km2
marine offshore
in EEZ plus
93,300 km2
marine offshore
in Prince Edward
Islands EEZ

Percentage Provincial Every 4 years Counts of SANBI and 60% of

of threatened conservation (monitoring threatened species Botanical threatened plant
species authorities, processes (IUCN Red List) Society of species by 2020
conserved South Africa’s being South Africa
ex situ Scientific developed
Authority by 2020)

Percentage SANBI Every 4 years Reported DEA, with 1% of plant

of species support species by 2020
with ex situ from SANBI’s
collections active zoological
in restoration and biological
programmes gardens

Threat status of Provincial National Four datasets SANBI Minimum 60% of

ecosystems conservation Biodiversity (ecosystem types, each ecosystem
authorities, Assessments ecological conditions, type in good
DEA, DAFF, are updated protected areas, ecological
CSIR, research every 7 years biodiversity targets); condition
institutions local data sets where
possible, otherwise
global with some
ground truthing
 Part V: Monitoring and reporting 145

BOX 12.2, continued

Defining indicators and a monitoring plan in South Africa

TABLE 12.3, continued

Example of a monitoring template for a biodiversity policy in South Africa

Responsible
Source of Monitoring Measurement entity or Goal value for
Indicator data frequency method institution year Y

Protection level Provincial National As above SANBI Minimum 20% of

of ecosystems conservation Biodiversity each ecosystem
authorities, Assessments
South Africa’s are updated
Scientific every 7 years
Authority

Benefit sharing: International Every year Desktop review DEA By 2025,

patents that patent registry; benefit-sharing
exist for agreements agreements
products made registered exist for
from local under South patents that are
biodiversity, or Africa’s commercialized.
that use local Bioprospecting, Benefit-sharing
or indigenous Access and agreements
knowledge, Benefit-Sharing have been
and that have Regulatory reviewed
benefit-sharing Framework
agreements

Percentage of All national, Every 5 years Reporting progress Presidency By 2020, 100%
SDFs, integrated provincial and on the Mid Term of SDFs include
development municipal Strategic Framework maps for critical
plans and land- departments biodiversity
use schemes responsible for areas and
that include development control
biodiversity planning and development
considerations monitoring;
Department
of Rural
Development
and Land
Reform

Increase StatsSA Every year NBES DEA By 2030,

in average 10% increase
annualized compared with
GDP growth 2020
rate of the
South African
bioprospecting
and wildlife
sectors

Abbreviations: CSIR, Council for Scientific and Industrial Research; DAFF, Department of Agriculture, Forestry and Fisheries;
DEA, Department of Environmental Affairs; EEZ, exclusive economic zone; IUCN, International Union for Conservation of Nature;
NBES, National Bio-Economy Strategy; SANBI, South African National Biodiversity Institute; SDF, spatial development framework;
Stats SA, Statistics South Africa
146 Sustainable Development Methodology

12.6 Monitor indicators over time both an individual policy and broader national goals.
Box 12.3 shows an example of developing a plan to
Once indicators and parameters have been defined, monitor progress towards SDGs by cities in Bolivia.
it is a key recommendation to monitor each of
the indicators over time, in accordance with the Across the 169 targets defined for the 17 SDGs,
monitoring plan. Indicators should be monitored in there are a mix of quantitative targets (e.g. Goal 3,
relation to historical values, goal values and values at Target 3.1: “By 2030 reduce the global maternal
the start of policy implementation to understand the mortality ratio to less than 70 per 100,000 live births”)
performance of the policy over time. and qualitative targets (e.g. Goal 15, Target 15.9: “By
2020, integrate ecosystem and biodiversity values
It is a key recommendation to separately monitor into national and local planning, development
indicators for different groups in society, where processes …”). Therefore, indicators should be
relevant. Examples of different groups are men and defined either quantitatively or qualitatively,
women, people of different income groups, racial depending on the target.
or ethnic groups, people of different education
levels, people from different geographic regions, Although top-down national statistics and indicators
and people in urban versus rural locations. This are useful to monitor overall country progress
allows users to understand distributional impacts towards SDGs, progress towards achieving the SDGs
on different groups, and manage trade-offs in cases is made by implementing policies on the ground. To
where policies have positive impacts on some groups ensure that these policies are effective, a national
and negative impacts on others. Users should report MRV system should be established to collect data
distributional impacts on different groups to identify relating to individual policies, and their impact and
and manage potential trade-offs. effectiveness should be assessed using the previous
sections in this methodology.
If monitoring indicates that the assumptions used
in the ex-ante assessment are no longer valid, Box 12.4 shows an example of identifying SDG
users should document the differences and take targets and indicators that are relevant to a policy
the monitoring results into account when updating assessed in Kenya, which can help link the results
the ex-ante estimates or when estimating impacts of a policy assessment with monitoring progress
ex-post. Users should also determine whether towards SDGs.
the assumptions on key indicators in the ex-ante
assessment (from Chapters 8 and 9) remain valid.

12.7 Track progress towards SDGs

In addition to monitoring progress of individual

policies (described in previous sections), users may
also want to track overall progress towards SDGs
and/or related national or subnational sustainable
development goals, especially goals related to the
policy assessed. Tracking national progress, for
example, involves defining national indicators for
each goal and tracking progress of these indicators
over time by comparing historical values (if data are
available) to desired goal values in a future year.

Many countries are developing their own national

implementation plans, and in the process selecting
targets, indicators and methodologies. In principle,
targets, indicators and methods used to track
progress towards SDGs should be aligned with those
used for existing and emerging national frameworks,
and, as far as possible, with those used for NDCs.
Table 12.4 provides illustrative examples of selecting
national indicators for tracking progress, relating to
TABLE 12.4

Examples of indicators that may be used by a country to track progress towards SDGs

Responsible
Examples of Examples of Source of Monitoring Measurement entity or Historical Target
goals corresponding targets Indicator data frequency method institution value value

Examples of SDGs relating to a renewable energy policy

SDG 3: Ensure Target 3.8: Achieve universal Number of Survey Annual Community- Health 75 250
healthy lives and health coverage, including health clinics level Ministry
promote well- financial risk protection, electrified assessment
being for all at all access to quality essential
ages health-care services and
access to safe, effective,
quality and affordable
essential medicines and
vaccines for all

SDG 5: Achieve Target 5.5: Ensure Share of Survey Annual Community- Ministry of 10 30
gender equality women’s full and effective female level Social Affairs
and empower all participation and equal entrepreneurs assessment
women and girls opportunities for leadership (%)
at all levels of decision-
making in political,
economic and public life

SDG 7: Ensure Target 7.1: By 2030, Share of Survey Annual Community- Ministry of 58 85
access to ensure universal access people with level Energy
affordable, to affordable, reliable and access to assessment
reliable, modern energy services electricity
sustainable and services (%)
modern energy
for all

SDG 8: Promote Target 8.5: By 2030, Share of Survey Annual Community- Ministry of 65 85
sustained, achieve full and productive people (men/ level Social Affairs
inclusive and employment and decent women) in assessment
sustainable work for all women and jobs
economic growth, men, including for young
full and productive people and persons with
employment and disabilities, and equal pay
Part V: Monitoring and reporting 147

decent work for all for work of equal value

TABLE 12.4, continued

Examples of indicators that may be used by a country to track progress towards SDGs

Responsible
Examples of Examples of Source of Monitoring Measurement entity or Historical Target
goals corresponding targets Indicator data frequency method institution value value

Examples of other SDGs in a country

SDG 2: End Target 2.3: By 2030, double Rice yield National rice Annual Combined Ministry of 2125 kg/ha 2700 kg/ha
hunger, achieve the agricultural productivity growth (kg/ha) information remote-sensing/ Agriculture in 2010 by 2020
food security and the incomes of small- system crop modelling
148 Sustainable Development Methodology

and improved scale food producers approaches

nutrition and
promote
sustainable
agriculture

SDG 3: Ensure Target 3.1: By 2030 reduce Reduction in Survey, civil Annual Large Health 300 in 2010 50 by 2030
healthy lives and the global maternal the national registration population- Ministry
promote well- mortality ratio to less than maternal systems based surveys,
being for all at all 70 per 100,000 live births mortality rate counting
ages

SDG 6: Ensure Target 6.1: By 2030, Proportion Survey Annual Large Health 75% in 2015 100% by
availability and achieve universal and of population population- Ministry 2030
sustainable equitable access to safe and that has based surveys
management affordable drinking water access to a
of water and for all sustainable,
sanitation for all safe water
supply and
hygienic
sanitation in
the household

SDG 7: Ensure Target 7.2: By 2030, Share of National Annual Calculation Ministry of 65% in 2016 85% by 2027
access to increase substantially the renewable energy based on MW Energy
affordable, share of renewable energy energy in information of renewable
reliable, in the global energy mix national system energy installed
sustainable and energy mix
modern energy
for all
TABLE 12.4, continued

Examples of indicators that may be used by a country to track progress towards SDGs

Responsible
Examples of Examples of Source of Monitoring Measurement entity or Historical Target
goals corresponding targets Indicator data frequency method institution value value

SDG 9: Build Target 9.1: Develop quality, National National Once (in Presence/ Ministry of In 2014, By 2018,
resilient reliable, sustainable and Construction Construction 2018) absence of Construction National National
infrastructure, resilient infrastructure, Code for Code features on Construction Construction
promote inclusive including regional and buildings takes extreme Code for Code for
and sustainable trans-border infrastructure, into account wind events buildings buildings
industrialization to support economic extreme wind in National does not includes
and foster development and human events Construction take into features on
innovation well-being, with a focus on Code for account extreme
affordable and equitable buildings extreme wind events
access for all wind events

SDG 15: Target 15.2: By Reduction National Annual Remote-sensing Ministry of Defore- Defore-
Protect, restore 2020, promote the in the environment modelling Agriculture/ station rate station rate
and promote implementation of deforestation statistics approaches Ministry of of 1.29% in of 0% by
sustainable use sustainable management rate Environment 2015 2030
of terrestrial of all types of forests, halt
ecosystems, deforestation, restore
sustainably degraded forests and
manage substantially increase
forests, combat afforestation and
desertification, reforestation globally
and halt and
reverse land
degradation and
halt biodiversity
loss
Part V: Monitoring and reporting 149
150 Sustainable Development Methodology

BOX 12.3

Monitoring progress towards SDGs by cities in Bolivia

Cities and local governments, in addition to other non-governmental stakeholders, are recognized as key implementers
of the SDGs as the core of the 2030 Agenda for Sustainable Development. The Cities Footprint Project in Bolivia has the
goal of promoting low-carbon and climate-resilient development in Latin American cities. In an assessment using the ICAT
Sustainable Development Methodology, Servicios Ambientales S.A. developed a monitoring plan for the Bolivian cities of
La Paz, Cochabamba, Santa Cruz, El Alto and Tarija. The aim was to initiate monitoring and reporting processes towards
the SDGs, which will inform the cities’ development efforts to achieve the SDGs. Table 12.5 provides an example of the
monitoring plan for one selected SDG goal (Goal 6); the complete SDG monitoring plan includes many different SDG
goals, targets and indicators. In Table 12.5, target values are still to be established by the municipal governments, and the
monitoring frequency is monthly.

TABLE 12.5

Example of SDG monitoring plan for cities in Bolivia for an SDG goal

SDGs or Corre- Level Responsible Measure-

other sponding of data entity or ment
goals targets Indicator collection Source of data institution method

Goal 6: Target 6.4: 6.4.1: General National Statistical WHO, Water-use

Ensure By 2030, Change in Sources Office (INE), UNICEF, efficiency is
availability substantially water-use particularly for Vice-Ministry defined as
and increase efficiency economic data. of Water and the value
sustain- water-use over time Administrative Irrigation added for a
able efficiency data collected given major
manage- across all at country level sector divided
ment of sectors by the relevant by the volume
water and and ensure institutions, of water
sanitation sustainable either technical used.
for all withdrawals (for water and The unit of
and supply of irrigation) or the indicator
fresh water to economic (for is value/
address water value added). volume
scarcity and These data are (commonly
substantially then compiled by $/m3).
reduce the FAO, World Bank,
Services’
number UNSD and other
water supply
of people international
efficiency is
suffering from institutions;
calculated as
water scarcity harmonized;
the service
and published
sector value
in sectoral
added divided
databases such as
by water
AQUASTAT (FAO),
used for
Databank (World
distribution
Bank) and UNdata
by the water
(UNSD).
collection,
City of La Paz Report from Municipal treatment
Public Social government and supply
Enterprise of water and industry,
Water and sanitation expressed in
Sanitation of La directorate $/m3.
Paz (EPSAS) (EPSAS)
 Part V: Monitoring and reporting 151

BOX 12.3, continued

Monitoring progress towards SDGs by cities in Bolivia

TABLE 12.5, continued

Example of SDG monitoring plan for cities in Bolivia for an SDG goal

SDGs or Corre- Level Responsible Measure-

other sponding of data entity or ment
goals targets Indicator collection Source of data institution method

Goal 6, Target 6.4, 6.4.1, City of Report from Municipal

continued continued continued Cochabamba Cochabamba government
Municipal Service water and
of Drinking Water sanitation
and Sanitation directorate
(SEMAPA) (SEMAPA)

City of Santa Report from Municipal

Cruz Drinking Water government
and Sanitary water and
Sewer Service sanitation
(SAGUAPAC) directorate
(SAGUAPAC)

City of El Alto Report from Municipal

Public Social government
Enterprise of water and
Water and sanitation
Sanitation of directorate
El Alto (EPSAS) (EPSAS)

City of Tarija Report from Municipal

Co-op for Water government
Services and water and
Sanitation Tarija sanitation
(COSSALT) directorate
(COSSALT)

Abbreviations: UNICEF, United Nations Children’s Fund; UNSD, United Nations Statistics Division; WHO, World Health Organization

BOX 12.4

Identifying SDG targets and indicators relevant to a policy assessed in Kenya

UNEP DTU Partnership conducted an ex-ante assessment of the sustainable development impacts of a policy to promote
solar PV mini-grids in Kenya. Ten impact categories were assessed qualitatively, ranging from accessibility and quality of
health care to gender equality and empowerment of women. Four impact categories were assessed quantitatively: climate
change mitigation, air pollution, human toxicity and resources depletion. To identify SDG indicators that are relevant to
the solar PV mini-grid policy, the study first identified SDG targets that are directly connected with the impact categories
and specific impacts analysed in the assessment. The study explains the reason why the assessed impact categories are
connected with specific SDG targets (Table 12.6).
152 Sustainable Development Methodology

BOX 12.4, continued

Identifying SDG targets and indicators relevant to a policy assessed in Kenya

TABLE 12.6

Examples of linkages between impact categories and SDG targets for the solar PV mini-grid
in Kenya

Impact category
assessed SDG target Rationale

Climate change 9.4: By 2030, upgrade infrastructure and retrofit 9.4: The impact on climate change
mitigation industries to make them sustainable, with mitigation of the policy increases
increased resource-use efficiency and greater resource-use efficiency. It consists of
adoption of clean and environmentally sound adoption of clean and environmentally
technologies and industrial processes, with all sound technology.
countries taking action in accordance with their 13.2: The policy is a climate change
respective capabilities measure.
13.2: Integrate climate change measures into
national policies, strategies and planning

Accessibility and 3.8: Achieve universal health coverage, including 3.8 The specific impacts of “Improved
quality of health care financial risk protection, access to quality access to health care due to better
essential health-care services and access to service in health centres and longer
safe, effective, quality and affordable essential working hours” and “Improved access
medicines and vaccines for all to health care due to the possibility of
storing vaccines” are connected with
accessing quality essential health-care
services and vaccines for all.

Gender equality and 5.6: Ensure universal access to sexual and 5.6 Through the specific impact
empowerment of reproductive health and reproductive rights “Knowledge on health and family
women as agreed in accordance with the Programme planning”, the action will support access
of Action of the International Conference to sexual and reproductive health.
on Population and Development and the 11.7 By increasing “Mobility at dark
Beijing Platform for Action and the outcome hours”, the action will provide access
documents of their review conferences to safer public spaces, particularly for
11.7: By 2030, provide universal access to women and children.
safe, inclusive and accessible, green and public
spaces, in particular for women and children,
older persons and persons with disabilities

Based on the identified SDG targets, a list of relevant indicators for each target can be found in the United Nations Global
SDG Indicators Database as well as Kenya’s National SDG Indicator Framework (a nationally defined subset of indicators
agreed to be tracked in the country). The United Nations Global SDG Indicators Database also provides historical values for
these indicators in Kenya. Using these indicators, progress can be tracked towards specific SDG targets.
13 Reporting

Reporting the results, methods and assumptions used Chapter 2: Objectives

is important to ensure that the impact assessment • The objective(s) and intended audience(s) of
is transparent, and gives decision makers and the assessment
stakeholders the information they need to properly
interpret the results. This chapter presents a list of Chapter 3: Overview of key concepts and steps
information that is recommended to be reported. This • Whether the assessment consists of a
information can also be useful to inform reporting qualitative impact assessment, a quantitative
under the Paris Agreement.47 impact assessment and/or tracking progress
of indicators over time
Checklist of key recommendations
• Opportunities for stakeholders to participate
• Report information about the assessment in the assessment
process and the sustainable development
impacts resulting from the policy (including Chapter 4: Describing the policy
the information listed in Section 13.1) • A description of the policy, including the
recommended information in Table 4.1

• Whether the assessment applies to an

13.1 Recommended information individual policy or a package of related
to report policies; if a package is assessed, which
policies are included in the package
It is a key recommendation to report information
about the assessment process and the sustainable • Whether the assessment is ex-ante, ex-post,
development impacts resulting from the policy or a combination of ex-ante and ex-post
(including the information listed below). For guidance
on providing information to stakeholders, refer to Chapter 5: Choosing which impact categories
the ICAT Stakeholder Participation Guide (Chapter 7). and indicators to assess
• A list of impact categories included and
General information excluded from the assessment boundary,
• The name of the policy/action assessed with justification for exclusions of impact
categories that may be relevant, significant or
• The person(s) or organization(s) that did the identified by stakeholders
assessment
• Indicator(s) selected for each impact category
• The date of the assessment included in the assessment boundary

• Whether the assessment is an update of a Chapter 6: Identifying specific impacts within

previous assessment, and, if so, links to any each impact category
previous assessments • A list of all sustainable development impacts
identified, using a causal chain and/or table
format

Chapter 7: Qualitatively assessing impacts

• The assessment period
47
For example, when providing information necessary to track
progress on the implementation and achievement of policies
and measures implemented to address the social and economic • A description of each specific impact
consequences of response measures (paragraph 78 of the
modalities, procedures and guidelines for the transparency
• The outcomes of the qualitative assessment
framework for action and support referred to in Article 13 of the
Paris Agreement). for each impact (including likelihood,
154 Sustainable Development Methodology

magnitude and whether it is positive or such as annually and cumulatively over the
negative), including which identified impacts assessment period, if feasible
are significant, and the methods and sources » The total in-jurisdiction impact and,
used separately, the total out-of-jurisdiction
impact, for each indicator, if relevant and
• A summary of the qualitative assessment feasible
results for each impact category, including » Justification for why any impacts in the
impacts of the policy on different groups in assessment boundary have not been
society, where relevant estimated, with a qualitative description of
the impacts
Chapter 8: Estimating the baseline » The assessment methods used
• For users following a quantitative approach: » A description of the policy scenario for
each indicator being estimated
» A list of impacts and indicators included » The policy scenario values for each
in the quantitative assessment boundary indicator being estimated, and the
and a list of any impacts that are not methods, assumptions and data sources
quantified, with justification used to calculate policy scenario values
» A description of the baseline scenario » Distributional impacts on different groups
for each indicator being estimated and a in society
justification for why it is considered to be
the most likely scenario Chapter 10: Estimating impacts ex-post
» The methods, assumptions and data used • For users estimating impacts ex-post:
to estimate the baseline scenario for each
indicator being estimated, including the » The estimated net impact of the policy, for
source of the baseline scenario if adapted each indicator, over defined time periods,
from a previous analysis such as annually and cumulatively over the
» The baseline values for each indicator assessment period, if feasible
being estimated over defined time periods, » The total in-jurisdiction impact and,
such as annually over the assessment separately, the total out-of-jurisdiction
period, if feasible impact, for each indicator, if relevant and
» The methods, assumptions and data feasible
sources used to calculate baseline values » Justification for why any impacts in the
» A list of policies, actions and projects assessment boundary have not been
included in each baseline scenario, with estimated, with a qualitative description of
justification for any implemented or the impacts
adopted policies, actions or projects with » The assessment methods used
a potentially significant impact that are » The policy scenario values for each
excluded from a baseline scenario indicator being estimated, and the
» A list of non-policy drivers included in each methods, assumptions and data sources
baseline scenario, with justification for any used to calculate policy scenario values
relevant non-policy drivers excluded from » Distributional impacts on different groups
a baseline scenario in society
» Which planned policies are included in the
baseline scenario, if any Chapter 11: Assessing uncertainty
» Justification for the choice of whether • The method or approach used to assess
to estimate new baseline values and uncertainty
assumptions or to use published baseline
values and assumptions • A quantitative estimate or qualitative
» If it is not possible to report a data source, description of the uncertainty and sensitivity
justification for why a source is not of the results, to help users of the information
reported properly interpret the results

Chapter 9: Estimating impacts ex-ante Chapter 12: Monitoring performance over time
• For users estimating impacts ex-ante: • A list of indicators used to track progress over
time and the rationale for their selection
» The estimated net impact of the policy, for
each indicator, over defined time periods,
 Part V: Monitoring and reporting 155

• Sources of indicator data and monitoring • Historical values for the indicators included in
frequency the assessment

• The performance of the policy over time, as • Sustainable development goals of the
measured by the indicators, and whether the implementing jurisdiction
performance of the policy is on track relative
to expectations • The contribution of the assessed policy
towards the jurisdiction’s sustainable
• Whether the assumptions on key indicators development goals
within the ex-ante assessment remain valid, if
applicable • How the policy is modifying longer-term
trends
• Trends in indicators for different groups in
society • Any potential overlaps with other policies

• Any limitations in the assessment not

described elsewhere
13.2 Additional information to report
(if relevant) • The type of technical review undertaken (first,
second or third party), the qualifications of the
• The impact of the policy on different groups reviewers and the review conclusions (further
in society, such as men and women, people of guidance on reporting information related to
different income groups, people of different technical review is provided in Chapter 9 of
racial or ethnic groups, people of different the ICAT Technical Review Guide)
education levels, people from different
geographic regions, and people in urban • Other relevant information
versus rural locations
Box 13.1 provides an example of how the
• A range of likely values for the net change in assessment results can be used to report progress
each indicator, rather than a single estimate, made in achieving SDGs for a country.
when uncertainty is high

BOX 13.1

Using the assessment results to report progress towards SDGs in Kenya

UNEP DTU Partnership conducted an ex-ante assessment of the sustainable development impacts of a policy to promote
solar PV mini-grids in Kenya. Ten impact categories were assessed qualitatively, ranging from accessibility and quality of
health care to gender equality and empowerment of women. Four impact categories were assessed quantitatively: climate
change mitigation, air pollution, human toxicity and resources depletion.
One objective of the study was to link the policy’s impacts to progress in achieving the SDGs. Similar to the case study shown
in Box 12.4, the first step was to link specific impacts identified in the assessment with SDG targets. The study then used two
different approaches: one for qualitatively assessed impacts and one for quantitatively assessed impacts.
For qualitatively assessed impacts, the study used the colour coding in Figure 13.1 to classify each impact as having a very
negative, negative, uncertain/insignificant, positive or very positive impact on each SDG target.
156 Sustainable Development Methodology

BOX 13.1, continued

Using the assessment results to report progress towards SDGs in Kenya

FIGURE 13.1

Colour coding scheme for qualitative impacts

Magnitude (negative) Magnitude (positive)

Score Major Moderate Minor Minor Moderate Major

Very likely

Likely
Likelihood

Possible

Unlikely

Very unlikely

Very negative impact Negative impact Uncertain/insignificant impact Positive impact Very positive impact

For quantitatively assessed impacts, the study calculated the relative improvement for each impact category by using either
equation 13.1 or equation 13.2, depending on the impact category. For each impact category, an indicator was defined, such
as PM2.5 (t/year) for air pollution and CO2e (kg/year) for climate change mitigation. The study then used Figure 13.2 to classify
each impact as having a very negative, negative, uncertain/insignificant, positive or very positive impact on each SDG target,
based on the results of the equation.

Equation 13.1: For impact categories where the goal is to increase the indicator value (e.g. jobs)

Policy scenario impact – Baseline scenario impact

Relative improvement (%) =
Baseline scenario impact

Equation 13.2: For impact categories where the goal is to decrease the indicator value (e.g. air pollution)

Baseline scenario impact – Polio scenario impact

Relative improvement (%) =
Baseline scenario impact

Note: The equations can be applied either annually or cumulatively over the assessment period.

FIGURE 13.2

Colour coding scheme for quantitative impacts

VERY NEGATIVE NEGATIVE UNCERTAIN OR INSIGNIFICANT IMPACT POSITIVE VERY POSITIVE

IMPACT IMPACT IMPACT IMPACT

–100% –66% –33% 0% +33% +66% +100%

 Part V: Monitoring and reporting 157

BOX 13.1, continued

Using the assessment results to report progress towards SDGs in Kenya

The study then used Figure 13.3 to give a visual representation of the policy’s impacts on the various SDG targets, combining
both the qualitative and quantitative results. The figure shows where the policy has a positive, negative or uncertain impact
on the various SDG targets. The individual circles in the SDG boxes represent the 169 SDG targets.

FIGURE 13.3

Impacts of the policy on the SDG targets

1 2 3 4 5 A B 1 2 3 4 5 A B 1 2 3 4 5 6 7 8 9 A B C D

1 2 3 4 5 6 7 A B C 1 2 3 4 5 6 A B C 1 2 3 4 5 6 A B

1 2 3 A B 1 2 3 4 5 6 7 8 9 10 A B 1 2 3 4 5 A B C

1 2 3 4 5 6 7 A B C 1 2 3 4 5 6 7 A B C 1 2 3 4 5 6 7 8 A B C

1 2 3 A B 1 2 3 4 5 6 7 A B C 1 2 3 4 5 6 7 8 9 A B C

1 2 3 4 5 6 7 8 9 10 A B 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

Very negative impact Negative impact Uncertain/insignificant impact Positive impact Very positive impact
158 Sustainable Development Methodology

BOX 13.1, continued

Using the assessment results to report progress towards SDGs in Kenya

Additionally, the study used Figure 13.4 to report the quantitative results as relative improvements in each SDG target,
based on the results of equations 13.1 and 13.2.

FIGURE 13.4

Quantified impacts of the policy on SDG targets

100%
AVERAGE RELATIVE IMPROVEMENT PER YEAR

80%

60%

40%

20%

0%

–20%

–40%

–60%

–80%

–100%

–120%

–140%
3.9 9.4 11.6 12.2 13.2

SDG TARGETS
PART VI
Decision-making and using results
14 Evaluating synergies and trade-offs,
and using results

This chapter provides an overview of approaches for • Efficiency or cost-effectiveness. Which

understanding and evaluating the results and possible policy option generates the greatest positive
trade-offs across multiple impact categories included impacts for a given level of resources?
in the assessment, and making decisions based on the
results. The chapter is applicable to qualitative and • Coherence. Which policy option is most likely
quantitative assessments, either ex-ante or ex-post. to avoid negative impacts, limit trade-offs
and achieve net benefits across the various
impact categories that are relevant to policy
objectives?
14.1 Introduction to approaches
The same questions can be asked of different
After assessing the impacts of a policy on the various policy design or implementation choices within
impact categories in previous chapters, the final a single policy option, to optimize policy design
step is to evaluate the results across all the impact and implementation. During or after policy
categories and draw conclusions to make decisions implementation, the same questions can also be
about policy selection, design and implementation. asked to determine how effective policies have
In many cases, users will need to evaluate trade-offs, been, to inform any adjustments to policy design
since the policy is likely to achieve positive benefits in or implementation and decide whether to continue
some impact categories and have negative effects in current actions, enhance current actions or
others. implement additional actions.

Policies can be evaluated based on the following Multiple methods are available to address these
criteria to determine which to implement or questions. This chapter focuses on three such
prioritize:48 methods (summarized in Table 14.1):

• Effectiveness. Which policy option maximizes • cost-effectiveness analysis (CEA)

positive impacts and achieves desired
outcomes across selected impact categories, • cost–benefit analysis (CBA)
and best contributes to broader goals such as
SDGs? • multi-criteria analysis (MCA).

FIGURE 14.1

Overview of steps in the chapter

Apply CEA, CBA

Introduction to Assess uncertainty Use results to make
and/or MCA
approaches and sensitivity decisions
(Sections 14.2,
(Section 14.1) (Section 14.5) (Section 14.6)
14.3, 14.4)

48
European Commission (2009).
 Part VI: Decision-making and using results 161

TABLE 14.1

Summary of methods

Method Description Advantages Disadvantages

Cost- • Determines the ratio of costs to Simple approach; does not Results in multiple
effectiveness effectiveness for a given impact category require that non-monetary indicators when
analysis • Can be used to compare policy options benefits be quantified in assessing more than
to determine which is most effective in monetary terms; fewer one impact category;
achieving a given objective for the least subjective elements requires discount rates
cost

Cost–benefit • Determines the net benefits to society Assesses aggregated benefits Complex approach that
analysis (the difference between total social (across the environmental, requires monetizing
benefits and total social costs) of policy social and economic non-monetary costs
options dimensions) of policy options and benefits, and
• Can be used to compare policy options with one single indicator requires discount rates;
to determine which has the greatest net can underestimate
benefit to society, or to analyse a single non-monetary benefits
policy to determine whether its total
benefits to society exceed its costs

Multi-criteria • Compares the favourability of policy Assesses aggregated benefits Has significant
analysis options based on multiple criteria (across the environmental, subjective elements
• Can be used to determine the most social and economic
preferred policy option dimensions) of policy options
with one single indicator;
does not require that
non-monetary benefits be
quantified in monetary terms;
does not require discount rate

Users should select one or more methods based on number of jobs created, or number of people with
the objectives and circumstances. CEA and CBA are increased access to energy. Expressing these impacts
relevant to quantitative impact assessments, since in monetary terms is useful to carry out a CBA, but is
they both require estimates of policy impact, whereas not always necessary to understand the benefits and
MCA can be applied to either qualitative or quantitative costs arising from a policy, and make decisions about
impact assessment. CBA and MCA are best suited to which policies to implement.
assessing multiple impact categories, whereas CEA
works well if the policy has one primary objective and Users should define the impacts that are included in
one primary measure of effectiveness (although it the CEA, CBA or MCA in a way that avoids duplication
can be used to provide multiple results – one for each and overlap between impacts. Defining distinct
impact category). CEA and MCA are easier to conduct impacts helps avoid double counting, which could
than CBA, which requires more complex techniques lead to biased results.
such as monetizing impacts. Other approaches beyond
CEA, CBA and MCA include life cycle cost assessment
and economic rate of return.
14.2 Cost-effectiveness analysis
Valuing or monetizing impacts is not always
necessary when assessing the impacts of a policy. CEA involves comparing different policy options
The method outlined in Parts II, III and IV explain based on their costs in achieving a single desired
how to quantify the impacts of policies in physical objective. The output of a CEA is a ratio of costs to
terms, such as tonnes of air pollution reduced, effectiveness for a given policy option, such as cost
162 Sustainable Development Methodology

per job created or cost per tonne of air pollution costs) and indirect costs to members of society
reduced. This ratio can be compared across policy (e.g. higher fuel prices). Users should include direct
options to determine which is most cost-effective. government costs in all cases. Depending on the
Cost-effectiveness can also be calculated for different purpose of the analysis, users can include other
groups in society to assess distributional impacts. monetary costs when conducting the CEA. There
may also be negative costs that should be taken into
In general, a CEA consists of three steps: account – that is, monetary costs that are reduced
because of the policy, such as reduced energy costs
1. Estimate the cost of each policy option. or reduced subsidies for fossil fuel.

2. Estimate the impact of each policy option for Users should compare costs of different policy options
relevant impact categories. based on the present value of costs. Costs that are
incurred over time can be converted to present value
3. Calculate the cost-effectiveness of each policy by applying a discount rate. Equation 14.1 provides
option for relevant impact categories. equations for calculating the present value of costs.
Box 14.1 provides more information on discount
rates. Table 14.2 provides an example of calculating
14.2.1 Step 1: Estimate the cost of each costs for two illustrative policies over a 10-year period.
policy option
Equation 14.1: Calculating present value of costs
In CEA, cost refers to monetary costs. The cost of
n
policy options could include direct costs to the PVC = ∑t=0 Ct / (1 + r)t
government to implement the policy (e.g. budget
expenditure and administrative costs), direct costs to where PVC is the present value of costs, Ct is costs in a
members of society (e.g. taxes and other compliance particular year, r is the discount rate, t is the number

BOX 14.1

Discount rates

Costs and benefits are likely to arise over multiple time periods. In economic theory, monetary impacts in the future are
worth less to individuals than resources available today, since individuals can earn a return on investment on money they
possess today, which they forego when receiving the same amount of money in the future. Both CEA and CBA typically
convert monetary values to their present value by using a discount rate.
For sustainable development impacts, social discount rates are most appropriate, since they reflect a society’s relative
valuation of today’s well-being versus well-being in the future. Social discount rates can vary widely – for example, from
0% to more than 10% – depending on how they address equity concerns with respect to future generations, among other
considerations not accounted for in national interest rates or typical discount rates. The World Bank has recommended
using social discount rates of 6% for low- and middle-income countries, and 4% for high-income countries.48 The European
Commission Impact Assessment Guidelines recommends a discount rate of 4%.49
The following discussion offers further perspectives on the choice of a discount rate: “A high discount rate suggests those
alive today are worth more than future generations. A third approach to discounting, based on ethics, says this is wrong, and
argues for a very low or even zero rate. This is why the Stern Review on the economics of climate change published in 2006
adopted a rate of 1.4%. US government guidance is to use discount rates of both 3% and 7% for valuing costs and benefits
within a single generation of, say, 30 years. It suggests using a lower rate, for time horizons that cross generations. UK
government guidance from HM Treasury is to use a 3.5% rate. However, it says: ‘The received view is that a lower discount
rate for the longer term (beyond 30 years) should be used.’ It sets out a sliding scale falling to 1% for time periods greater
than 300 years. In a major survey of 197 economists, the average long-term discount rate was 2.25%. The survey found
almost all were happy with a rate of between 1 and 3%, whereas only a few favoured higher figures.“50 Users should consider
a range of discount rates and conduct sensitivity analysis to see how the choice affects the overall results.

49
World Bank and IHME (2016).
50
European Commission (2009).
51
Carbon Brief (2017).
 Part VI: Decision-making and using results 163

of years from the present and n is the number of 14.2.3 Step 3: Calculate the cost-
years. effectiveness of each policy option for
relevant impact categories

14.2.2 Step 2: Estimate the impact of each Equation 14.2 provides the equation for calculating
policy option for relevant impact categories cost-effectiveness. Cost-effectiveness can only be
calculated for one impact category at a time. Users
Users should use the quantitative assessment can apply the method individually to each impact
results from previous chapters for all relevant category of interest to calculate different cost-
impact categories as the measure of impact for effectiveness ratios for each impact category, such as
each policy option – that is, the change in indicator cost per job created or cost per tonne of air pollution
value attributed to the policy. Table 14.3 provides reduced.
an illustrative example of the effectiveness of each
policy option. Equation 14.2: Calculating cost-effectiveness for a
policy
PVC
Cost-effectiveness =
         impact

TABLE 14.2

Example of calculating costs (present value) of two policies over a 10-year period
(illustrative results only)

Costs in each year (million $) Discounted costs (million $) Present

Dis- value
Policy count Year Year Year Year Year Year Year Year (million
options rate 1 2 … 9 10 1 2 … 9 10 $)

Solar PV 3% 1 1 … 1 1 0.97 0.94 … 0.77 0.74 8.53

incentive
policy

Energy 0.4 0.4 … 0.4 0.4 0.39 0.38 … 0.31 0.30 3.41
efficiency
policy

TABLE 14.3

Impact of two policies across three impact categories (illustrative results only)

Policy options GHG reduction Air pollution reduction Job creation

Solar PV incentive policy 50,000 tCO2e per year for 1,000 t PM2.5 per year for 200 jobs created in the first
10 years 10 years year, which last for 10 years

Energy efficiency policy 30,000 tCO2e per year for 600 t PM2.5 per year for 50 jobs created in the first
10 years 10 years year, which last for 10 years
164 Sustainable Development Methodology

TABLE 14.4

Calculating cost-effectiveness for a solar PV incentive policy (illustrative results only)

Policy options GHG reduction Air pollution reduction Job creation

Solar PV incentive policy $17 per tCO2e reduced $853 per t PM2.5 reduced $42,651 per job created

Energy efficiency policy $11 per tCO2e reduced $568 per t PM2.5 reduced $68,241 per job created

Note: Results are over the 10-year assessment period.

where PVC is the present value of costs, and impact is There are three steps to conducting a CBA:
the quantified change for a specific impact category.
1. Quantify all relevant costs and benefits of the
Table 14.4 shows the cost-effectiveness results policy.
for both policy options for each of three impact
categories: GHG reduction, air pollution reduction 2. Express non-monetary costs and benefits in
and job creation. In this illustrative example, the monetary terms.
energy efficiency policy is more cost-effective in
reducing GHG emissions and air pollution, but less 3. Calculate the present value of all cost and
cost-effective in creating jobs. benefits, and calculate the net present value
for each policy option.
From the point of view of cost-effectiveness, users
should balance the trade-offs and choose which
policy option to implement based on which impact 14.3.1 Step 1: Quantify all relevant costs
categories are most important and the relative and benefits of the policy
cost-effectiveness of the results. CBA and MCA offer
further approaches to help decide which policy In CBA, benefits refer to positive impacts and costs
option to implement. refer to negative impacts. Benefits also include
avoided negative impacts. Unlike CEA, where only
monetary costs are accounted for, CBA includes all
relevant social, economic and environmental costs
14.3 Cost–benefit analysis and benefits: both monetary and non-monetary.
Costs should be calculated as described for CEA,
Unlike CEA, CBA takes into account a wide variety while the broader impacts should be quantified in
of costs and benefits of a policy in an aggregated physical terms (rather than monetary terms), as
manner. CBA involves quantifying the benefits described in Parts II, III and IV. Table 14.5 provides
and costs of a policy, and expressing them in an example of costs and benefits for two policy
monetary terms, using valuation methods. These options.
amounts are used as a proxy to represent social and
environmental impacts that may not have an explicit
economic or monetary value.

The result of CBA can be used to determine whether

the net benefits of a single policy exceed its net
costs and therefore whether the policy should be
implemented (in the case of ex-ante assessment) or
continued (in the case of ex-post assessment). CBA
can also be used to compare multiple policy options
to determine which has the greatest net benefits to
society and should be implemented.
 Part VI: Decision-making and using results 165

TABLE 14.5

Costs and benefits of two policy options (illustrative results only)

Benefits

Air pollution
Policy options Costs GHG reduction reduction Job creation

Solar PV incentive policy $1,000,000 each 50,000 tCO2e per 1,000 t PM2.5 per 200 jobs created in the
year for 10 years year for 10 years year for 10 years first year, which last for
10 years

Energy efficiency policy $400,000 each 30,000 tCO2e per 600 t PM2.5 per year 50 jobs created in the
year for 10 years year for 10 years for 10 years first year, which last for
10 years

14.3.2 Step 2: Express non-monetary costs in the case of the solar PV incentive policy, the
and benefits in monetary terms monetary values for GHG reduction, air pollution
reduction and job creation are assumed to be
CBA involves expressing non-economic impacts $41/tCO2e, $140,000/t PM2.5, and $293,330/job,
in monetary terms using valuation methods. respectively, based on relevant literature.53 These
Economists estimate monetary values of non- values are illustrative and represent one of multiple
monetary costs and benefits by linking them to ways of assigning monetary values to benefits
market prices or quantifying their impact on utility, (e.g. estimating economic impacts of job creation).
such as the satisfaction a person derives from
consuming a particular good or their change in well-
being.52 14.3.3 Step 3: Calculate the present value of
all cost and benefits, and calculate the net
A downside of CBA is that many environmental and present value for each policy option
social benefits are intangible, uncertain, subjective
or controversial to monetize. If all costs and benefits The output of a CBA is a calculated value
cannot be properly quantified in monetary terms, representing the present value of net benefits of the
a partial CBA can be carried out that includes the policy to society. Users should discount the future
subset of costs and benefits that are quantified and costs and benefits to calculate the present value of
monetized. Alternatively, users can apply MCA, which costs and benefits, and calculate the net present
does not monetize benefits. value for each policy option. This step is similar to
step 1 for CEA. Users should use equation 14.3 to
Users should avoid double counting monetary values calculate the result, which is an aggregated value
across multiple impacts. For example, some policies representing the net present value of the net
to reduce GHG emissions also generate jobs, bringing benefits of the policy to society.
economic benefits, which may be reflected in the
monetary value of GHG reduction. If the benefit from The results can be used, for example, to determine
job creation is quantified separately from the benefit whether a policy has a positive net benefit to
from GHG reduction, the same benefit should not be society and therefore should be implemented, or
included in both monetary values. to compare two policy options and implement the
policy option with the greatest net benefits.
The appropriate monetary value for each impact
should be based on the specific circumstances
of the assessment. As an illustrative example,

53
Adapted from Interagency Working Group on Social Cost of
Greenhouse Gases (2016), U.S. EPA (no date, b) and Kentucky
52
European Commission (no date). Cabinet for Economic Development (2018).
166 Sustainable Development Methodology

n
CBA typically considers net benefits in aggregate PVC = ∑t=0 Ct / (1 + r)t
rather than addressing distributional impacts among
different groups in society. However, the various where PVC is the present value of costs, Ct is costs in a
costs and benefits in a CBA can be disaggregated particular year, r is the discount rate, t is the number
among different stakeholder groups to assess of years from the present and n is the number of
distributional impacts. Alternatively, if distributional years.
impacts are significant, MCA may be preferable.
Table 14.6 shows the calculation of net benefits of
Equation 14.3: Calculating the net benefit of a policy options for the illustrative solar PV incentive
policy policy, focusing on the monetized value of GHG
reduction, air pollution reduction and job creation. In
NPV = PVB – PVC the example, the solar PV incentive policy has greater
net benefits than the energy efficiency policy, so is
where NPV is the net present value, representing the the preferred policy option.
net benefits of the policy.
n
PVB = ∑t=0 Bt / (1 + r)t

where PVB is the present value of benefits, Bt is the

benefits in a particular year, r is the discount rate, t
is the number of years from the present and n is the
number of years.

TABLE 14.6

Calculation of net benefits (NPV) for two policy options (illustrative results only)

Discount Present value of

Policy options Annual costs/benefits rate Duration costs/benefits
10
Solar PV Costs $1,000,000 3% 10 years ∑t=1 $1,000,000 / (1+0.03)t =
incentive $8,530,203
policy
Benefits (50,000 × $41) + ∑t10
=1
$200,716,000/(1+0.03)t =
(1,000 × $140,000) + $1,712,148,193
(200 × $293,330) =
$200,716,000

Net $199,716,000 $1,712,148,193 – $8,530,203 =

benefits $1,703,617,990
10

Energy Costs $400,000 3% 10 years ∑t=1 $400,000 / (1+0.03)t =

efficiency $3,412,081
10
policy
Benefits (30,000 × $41) + ∑t=1 $99,896,500/ (1+0.03)t =
(600 × $140,000) + $852,137,408
(50 × $293,330) =
$99,896,500

Net $99,496,500 $852,137,408 – $3,412,081 =

benefits $848,725,327
 Part VI: Decision-making and using results 167

14.4 Multi-criteria analysis • What are the economic, social and political
factors that should be considered for the
MCA or multi-criteria decision analysis (MCDA) allows decision?
stakeholders to determine an overall preference
among alternative options, where the options Most questions in step 1 should be largely defined
accomplish multiple goals. It uses normalization and in the assessment steps detailed in Chapters 2, 4
weighting to aggregate results into one metric.54,55 and 5. Users should review these and determine
Indicators used to measure each criterion can be whether they are appropriate for the MCA. Users
qualitative or quantitative.56 There are multiple should also review whether the policy being assessed
ways to construct and apply an MCA. For example, creates appropriate options for the MCA, since an
different scales can be used to assign a performance MCA requires multiple policy options. If only a single
score and to determine criteria weight factors. policy’s sustainable development impacts are being
assessed, users should decide whether to conduct
This section provides simplified guidance based additional impact assessments for additional policy
on the MCDA approach described in the United options and/or use “no action” as an option.
Kingdom Government’s Multi-criteria Analysis: a
Manual.57 Additional references are listed at the end For example, in the case of a solar PV incentive
of the chapter for further guidance on this and other policy, the reason for the assessment is to support
MCA approaches. the government’s efforts to pursue multiple policy
objectives, such as addressing climate change,
MCA can be summarized into three general steps: improving health from improved air quality, creating
jobs, improving energy independence and reducing
1. Identify the decision context, policy options, budget deficits. Within that context, three policy
assessment objectives and criteria. options are identified: enact a solar PV incentive
policy, enact an energy efficiency policy, or take no
2. Score the performance of each policy option action. These policy objectives translate into five
for each criterion. criteria for the MCA: GHG reduction, air pollution
reduction, job creation, energy independence and
3. Assign a weight to each criterion, and calculate direct costs.
an overall score and/or cost–benefit ratio for
each option.
14.4.2 Step 2: Score the performance of each
policy option for each criterion
14.4.1 Step 1: Identify decision context,
policy options, assessment objectives and This step involves characterizing, either quantitatively
criteria or qualitatively, the performance of each option
against each criterion, then normalizing the
In the first step, the user should answer the following performance to scores.59
questions:58
A performance matrix can be used to summarize and
• What are the overall reasons or objectives for present the performance of options. For criteria that
the analysis and who are the stakeholders for are assessed quantitatively, the value should be used
the decision? directly. For criteria that are assessed qualitatively,
the user should provide a succinct description of the
• What are the options to be assessed? result.

• What is the decision that needs to be made? In the example of the solar PV incentive policy, four
criteria were quantified, and one criterion (energy
independence) was assessed qualitatively. The
results are shown in Table 14.7.

54
DCLG (2009).
The performance of each option should be assessed
relative to a baseline scenario (as described in
55
JISEA (2014).
Chapter 8). In this example, the baseline scenario is
56
WRI (2014).
57
DCLG (2009).
58
USAID (2014). 59
DCLG (2009).
168 Sustainable Development Methodology

“no action”, where no policy is implemented. When performance. However, such judgments are required
scoring the “no action” option, users should be to conduct an MCA for qualitatively assessed
aware that taking no action often also has costs. For criteria.60
example, not acting on climate change has significant
monetary, social, economic and environmental costs. Table 14.8 illustrates the performance scores for the
solar PV incentive policy.
After producing the performance matrix, users
should rank the performance for each criterion. For
criteria that are quantitatively assessed, the user 14.4.3 Step 3: Assign a weight to each
should assign 100 to the best option and 0 to the criterion, and calculate an overall score
worst option. All others should be scaled between and/or cost–benefit ratio for each option
these limits in proportion to their quantitative
impacts. In this step, users should determine how important
each criterion, or impact category, is to the decision.
For criteria that are assessed qualitatively, users can The process of deriving weights is fundamental to
directly assign scores to each option’s performance the effectiveness of MCA and has a very significant
for each criterion, giving the best performance a effect on the overall results.61 The weights should
score of 100 and the worst performance a score appropriately reflect value assumptions and policy
of 0, and score everything else in between. This may priorities. Since it is subjective, weighting should be
require making difficult judgments about the degree developed in consultation with stakeholders, such
of difference between each option’s qualitative as policymakers, businesses, civil society, and other

TABLE 14.7

Performance matrix for an illustrative multi-criteria analysis (illustrative results only)

GHG Air pollution Energy Monetary

Policy option reduction reduction Job creation independence costs ($)

Solar PV incentive policy 50,000 tCO2e 10,000 t PM2.5 200 Major positive impact 8,530,203

Energy efficiency policy 30,000 tCO2e 6,000 t PM2.5 50 Moderate positive 3,412,081
impact

No action 0 0 0 No impact 0

TABLE 14.8

Performance scores for an illustrative multi-criteria analysis (illustrative results only)

GHG Air pollution Job Energy Direct Monetary

Policy option reduction reduction creation independence costs ($)

Solar PV incentive policy 100 100 100 100 0

Energy efficiency policy 60 60 25 50 60

No action 0 0 0 0 100

60
DCLG (2009).
61
DCLG (2009).
 Part VI: Decision-making and using results 169

experts and affected stakeholders. Weighting should Equation 14.4: Calculating an overall score for
be guided by the objectives of the assessment, and each option
the local policy objectives and context. It should be

∑
transparently documented and justified. n
WjSij
j=1
One approach is to allocate a total of 100 points S=
among all criteria, with more points meaning that
i 100
the criterion is more important. When allocating   
the points, users should take into account the
importance of each criterion, and also the size of where Si is the overall score for option i, Wj is the
the difference between the least and most preferred weight for criterion j, and Sij is the performance score
options. For example, the user may decide that of option i for criterion j.
job creation is important, but, in the illustrative
case of the solar PV incentive and energy efficiency Table 14.9 shows the overall scores for each option
policies, the difference between the best- and in an illustrative MCA. In this example, the solar PV
worst-performing options is only 100 jobs, which is incentive policy has the highest score, so is the most
insignificant in the broader context of total jobs in a preferred policy option.
country. That criterion should receive a low weight
because the difference between the highest and Another useful approach is to calculate the benefits
lowest options is small.62 score without including monetary costs. To do so,
users should classify all criteria into two categories
Once the weights are determined, the user should – costs and benefits – assign weights to criteria in
determine an overall score for each option by the benefits category only, and then calculate the
calculating the weighted average of its scores on all weighted-average performance scores for each
the criteria.63 Equation 14.4 shows how to calculate option. By separating performance scores and costs,
the result. users can calculate the cost–benefit ratios for each
option.

Table 14.10 demonstrates how to calculate

performance scores and cost–benefit ratios. In
this example, the criteria weights from Table 14.9
have been scaled proportionately because direct

TABLE 14.9

Calculating overall scores for an illustrative multi-criteria analysis (illustrative results only)

Direct
GHG Air pollution Job Energy Monetary Overall
Policy option reduction reduction creation independence costs ($) score

Criteria weights 30 30 5 5 30 -

Solar PV incentive 100 100 100 100 0 70

policy

Energy efficiency 60 60 25 50 60 57.75

policy

No action 0 0 0 0 100 30

Abbreviation: -, not applicable

62
DCLG (2009).
63
DCLG (2009).
170 Sustainable Development Methodology

TABLE 14.10

Calculating performance scores for an illustrative multi-criteria analysis (illustrative results only)

Cost–
benefit
Direct ratio ($
Air Energy Overall monetary per unit of
Policy GHG pollution Job independ- perform- costs perform-
option reduction reduction creation ence ance score (million $) ance score)

Criteria
weights 42 42 8 8 - - -

Solar PV 100 100 100 100 100 8,530,203 85,302

incentive
policy

Energy 60 60 25 50 56.4 3,412,081 60,498

efficiency
policy

No action 0 0 0 0 0 0 -

Abbreviation: -, not applicable

monetary costs are now excluded. The solar PV differences in assumptions and values advocated
incentive policy has a higher cost–benefit ratio than by different stakeholders yield significantly different
the energy efficiency policy. If policymakers are results. If so, the assumptions and values should be
concerned with maximizing benefits or effectiveness, investigated and discussed further. If not, the results
the solar PV incentive policy is preferred, as shown can be considered more robust for purposes of
in Table 14.9. If policymakers are concerned with choosing between policy options.
maximizing benefits per unit of cost, the energy
efficiency policy is preferred. These results are very Table 14.12 shows how the values of key parameters
sensitive to assumptions about performance scores can be varied as part of a sensitivity analysis.
and criteria weights, so conclusions should be made Table 14.13 shows how a sensitivity analysis can be
carefully. calculated for one key parameter as part of a CEA.

14.5 Assess uncertainty

and sensitivity

All approaches to evaluating trade-offs (CEA, CBA

and MCA) involve a certain level of complexity and
subjectivity. Therefore, it can be useful to conduct
uncertainty and sensitivity analyses to examine the
extent to which key assumptions or different views
among stakeholders affect the results. Users should
follow the guidance in Chapter 11 to assess the
uncertainty and sensitivity of the results.

Table 14.11 provides examples of key parameters for

sensitivity analysis for CEA, CBA and MCA. The list is
not exhaustive, and users should consider whether
 Part VI: Decision-making and using results 171

TABLE 14.11

Examples of key parameters for sensitivity analysis

Type of analysis Key parameters for sensitivity analysis

Cost-effectiveness analysis Discount rate

Cost–benefit analysis Discount rate; monetary value of non-monetary costs and benefits

Multi-criteria analysis Criteria weights; performance scores for qualitatively assessed criteria

TABLE 14.12

Parameters considered for sensitivity analysis (illustrative results only)

Cost-
effectiveness
analysis Cost–benefit analysis Multi-criteria analysis

Criteria weights Performance

(GHG reduction : air scores for energy
Monetary pollution reduction : independence
value of CO2 job creation : energy (solar PV policy :
Sensitivity Discount rate Discount emissions independence : energy efficiency
scenario (%) rate (%) reduction ($) monetary costs) policy)

Primary 3 3 41 30:30:5:5:30 100:50

scenario

Alternative 1.4 1.4 13 10:40:5:5:40 100:20

scenario 1

Alternative 6 6 120 20:20:15:15:30 100:80

scenario 2

TABLE 14.13

Sensitivity analysis of discount rates in a cost-effectiveness analysis (illustrative results only)

GHG reduction Air pollution reduction Job creation

Sensitivity scenario Policy option ($ per tCO2e) ($ per t PM2.5) ($ per job)

Primary scenario: Solar PV incentive policy 17 853 42,651

discount rate 3% Energy efficiency policy 11 568 68,241

Alternative scenario 1: Solar PV incentive policy 19 927 46,356

discount rate 1.4% Energy efficiency policy 12 618 74,170

Alternative scenario 2: Solar PV incentive policy 15 736 36,800

discount rate 6% Energy efficiency policy 10 491 58,881
172 Sustainable Development Methodology

14.6 Using results to make decisions users can develop a performance matrix of policy
options (including no action), using effectiveness,
Depending on the assessment objectives, efficiency and coherence as criteria, as illustrated in
different decisions need to be made. For ex-ante Table 14.14. The example shows that any of these
assessments, decisions may include whether to policy options would be preferred based on certain
implement a specific policy, whether to implement criteria, but not on others. Users should prioritize
multiple policies, or how to improve a policy before or weight criteria to decide which policy option is
implementation. For ex-post assessments, decisions preferred overall.
may include whether to continue or discontinue a
policy that is in effect, whether to revive a policy that In some circumstances, rather than taking a neutral
is no longer in effect, or how to improve a policy approach to maximizing net benefits across all
during implementation. impact categories, users may want to focus on
minimizing negative impacts in certain key impact
categories or ensuring zero negative impacts across
14.6.1 Choosing a policy option all impact categories. Users should consider the
following factors when making decisions regarding
CEA, CBA and MCA provide useful insights on the trade-offs:
effectiveness, efficiency and coherence of policy
options. However, before decisions are made based • Minimum requirements. There may be
on the results, it is important to gather further inputs minimum thresholds for a given impact
and perspectives on the best course of action, since category below which a policy should not
each analytical approach has limitations and involves be implemented – for example, relating
subjective judgments. to human rights violations. Minimum
requirements are not negotiable, meaning
In general, policy options that do not have positive that the negative impact cannot be offset by
net benefits should be eliminated. The same is true positive impacts in other impact categories.
for policy options that are inferior to others under Minimum thresholds could be set by statutes,
every criterion. To assist with decision-making, science or sociopolitical expectations. In such

TABLE 14.14

Illustrative performance matrix for policy options (illustrative results only)

Policy
option Effectiveness Efficiency Coherence

Solar PV Reduces 50,000 tCO2e and $17 per tCO2e reduced; Good balance of climate, air,
incentive 10,000 t PM2.5; creates 200 jobs; $853 per t PM2.5 reduced; energy independence and
policy major positive impact on energy $42,651 per job created job impacts
independence (Table 14.7) (Table 14.4) Trade-off exists with
Overall performance score of 100 Cost of $85,302 per unit of monetary costs, but net
(Table 14.10) performance score (Table 14.10) benefits of $1,704 million
(Table 14.6)

Energy Reduces 30,000 tCO2e and 6,000 t $11 per t tCO2e reduced; Good balance of climate, air,
efficiency PM2.5; creates 50 jobs; moderate $568 per t PM2.5 reduced; energy independence and
policy positive impact on energy job impacts
$68,241 per job created
independence (Table 14.7) Trade-off exists with
(Table 14.4)
Overall performance score of 56.4 monetary costs, but net
Cost of $60,498 per unit of
(Table 14.10) benefits of $849 million
performance score (Table 14.10)
(Table 14.6)

No action No positive impacts No costs (or benefits) No trade-off (because there

are no benefits)

Source: Adapted from European Commission (2009).

 Part VI: Decision-making and using results 173

cases, users should either improve the policy combination of policies separately to determine
design to mitigate the negative impacts or which combination is best.
discontinue the policy option.

• Irreversibility. Policies may have negative 14.6.2 Improving policy design

impacts, such as loss of species, that are
irreversible, are deemed unacceptable and Users should consider improving policy design
cannot be offset with positive impacts in other based on the assessment results. In some cases,
impact categories. In such cases, users should the assessment findings may warrant complete
improve the policy design to avoid irreversible redevelopment of a policy option. To improve
negative impacts or discontinue the policy policy design, users can explore how different
option. policy implementation specifications can mitigate
any negative impacts. For example, if a solar PV
• Precaution. Policies may present major incentive policy is found to have negative impacts on
risks that are highly uncertain but could the national budget, policymakers can optimize the
be catastrophic. Users should adopt policy by choosing a financing model that would lead
the precautionary principle by taking to lower costs.
precautionary protection against potentially
hazardous impacts, and in such cases give Users should also consider establishing safeguards
more weight to avoiding negative impacts as part of the policy design (e.g. environmental
than achieving positive impacts.64 standards for solar manufacturing) to minimize
the likelihood of negative impacts, or developing
If multiple policy options are being considered for measures to offset any negative impacts (e.g. job
implementation, users should also be aware that, if retraining programmes for job losses in the coal-
policy A is better than policy B, it is not necessarily mining sector). The effectiveness of safeguards and
the case that policy A + C is better than policy B + C, offset measures should be evaluated and closely
because of the potential for interactions between monitored during the policy implementation period
the policies (described in Chapter 4). In such a case, to ensure that they are working as planned.65
users should consider evaluating the impact of each

TABLE 14.15

Further references on CEA, CBA and MCA

Reference Topics

Asian Development Bank (2007). Theory and Practice in the Choice of Social Discount Rate for Discount rates
Cost-Benefit Analysis: a Survey. Economics and Research Department Working Paper, Series
No. 94.

Bakhtiari, F. (2016). Valuation of Climate Change Mitigation Co-Benefits. Copenhagen: UNEP Valuation methods
DTU Partnership.

Boardman, A., and others (2006). Cost–Benefit Analysis: Concepts and Practice. Upper Saddle CBA
River, New Jersey: Prentice Hall.

Centre for European Policy Studies and Economisti Associati (2013). Assessing the Costs and CBA, discount rates,
Benefits of Regulation. Study for the European Commission, Secretariat General. valuation methods

Council of Economic Advisers (2017). Discounting for Public Policy: Theory and Recent Discount rates
Evidence on the Merits of Updating the Discount Rate.

Eureval-C3E (2006). Study on the Use of Cost-effectiveness Analysis in EC’s Evaluations. CEA

64
Federal Office for Spatial Development, Switzerland (2004). 65
Federal Office for Spatial Development, Switzerland (2004).
174 Sustainable Development Methodology

TABLE 14.15, continued

Further references on CEA, CBA and MCA

Reference Topics

European Commission (2009). Impact Assessment Guidelines. CEA, CBA, MCA, discount
rates

European Commission (2009). Impact Assessment Guidelines: Technical Annex. CEA, CBA, MCA, discount
rates

European Commission (2014). Guide to Cost–Benefit Analysis of Investment Projects. CBA

European Commission (no date). Better Regulation Toolbox. Chapter 8: Methods, models, CEA, CBA, MCA, discount
costs, and benefits. rates

HM Treasury, United Kingdom (2011). Green Book: Appraisal and Evaluation in Central CEA, CBA, MCA
Government.

Interagency Working Group on Social Cost of Greenhouse Gases, United States Government Social cost of carbon
(2016). Technical Update of the Social Cost of Carbon for Regulatory Impact Analysis: Under
Executive Order 12866.

Jeuland, Marc, and Jie-Sheng Tan Soo (2016). Analyzing the Costs and Benefits of Clean and CBA
Improved Cooking Solutions. Washington, D.C.: Clean Cooking Alliance.

Lawrence, Robert S., Lisa A. Robinson and Wilhelmine Miller, eds. (2006). Valuing Health for CEA
Regulatory Cost-Effectiveness Analysis. Chapter 5: Recommendations for regulatory cost-
effectiveness analysis. Washington, D.C.: National Academies Press.

Organisation for Economic Co-operation and Development (2006). Cost–Benefit Analysis and CBA
the Environment: Recent Developments.

Organisation for Economic Co-operation and Development (2014). OECD Regulatory CEA
Compliance Cost Assessment Guidance.

Organisation for Economic Co-operation and Development (2016). The Economic CBA
Consequences of Outdoor Air Pollution.

Puig, D., and S. Aparcana (2016). Decision-Support Tools for Climate Change Mitigation CEA, CBA, MCA
Planning. Copenhagen: UNEP DTU Partnership.

Scrieciu, S. Ş., and others (2014). Advancing methodological thinking and practice for MCA
development-compatible climate policy planning. Mitigation and Adaptation Strategies for
Global Change, vol. 19, No. 3, pp. 261–288.

United Kingdom Department for Communities and Local Government (2009). Multi-Criteria MCA
Analysis: a Manual.

United Kingdom Department for Environment, Food and Rural Affairs (2003). Use of Multi- MCA
criteria Analysis in Air Quality Policy: a Report.

United Nations Economic Commission for Europe (2017). Sustainable Development Briefs CBA
No.2: the Co-Benefits of Climate Change Mitigation.

United States Agency for International Development (2014). Application of Multi-Criteria MCA
Assessment (MCA) Methods: a Seven Step Process.

United States Environmental Protection Agency (2010). Guidelines for Preparing Economic CBA, valuation methods,
Analyses. discount rates
 Part VI: Decision-making and using results 175

TABLE 14.15, continued

Further references on CEA, CBA and MCA

Reference Topics

United States National Academies of Sciences, Engineering, and Medicine (2017). Valuing Social cost of carbon
Climate Damages: Updating Estimation of the Social Cost of Carbon Dioxide.

World Bank, Independent Evaluation Group (2007). Sourcebook for Evaluating Global and CEA, CBA, MCA
Regional Partnership Programs: Indicative Principles and Standards.

World Bank (2008). Social Discount Rates for Nine Latin American Countries. Washington, D.C. Discount rates

World Bank and ClimateWorks Foundation (2014). Climate Smart Development: Adding up the CBA, valuation methods,
Benefits of Actions that Help Build Prosperity, End Poverty and Combat Climate Change. discount rates

World Bank and Institute for Health Metrics and Evaluation, University of Washington (2016). CBA
The Cost of Air Pollution: Strengthening the Economic Case for Action.

World Health Organization (2003). WHO Guide to Cost-Effectiveness Analysis. CEA

APPENDICES
Appendix A: Example of quantifying
the impact of a solar PV incentive policy

This appendix provides an example of quantifying residential consumers) and 10 GW of grid-connected

the impact of a grid-connected rooftop solar PV tail-end plants. This example focuses on grid-
incentive policy. The example shows how to carry connected solar rooftop programmes that support
out an ex-ante assessment following the steps 40 GW installation by 2022.
outlined in Chapters 8 and 9 by developing an ex-
ante baseline and policy scenario, and estimating For previous steps related to the same example, see
the various sustainable development impacts of the Tables 4.1, 4.2, 5.2, 6.3, 7.5 and 8.1.
policy.
Chapter 8, Section 8.1: Define the quantitative
The Government of India has a target to achieve assessment boundary and period
100 GW solar capacity by 2022. The target is divided Table A.1 shows the set of impact categories, specific
into large-scale centralized power plants (50 GW) and impacts and indicators included in the quantitative
distributed smaller-scale projects: 40 GW of rooftop assessment boundary. The assessment period is
solar (mainly used by industrial, commercial and 2016–2025.

TABLE A.1

Impact categories, specific impacts and indicators included in the quantitative assessment boundary

Impact category Specific impacts Indicator to quantify

Climate change Reduced GHG emissions from grid- GHG emissions (tCO2e/year)
mitigation connected fossil fuel–based power plants

Air quality/health Reduced air pollution from grid- Emissions of PM2.5, PM10, SO2 and NOx (t/year);
impacts of air pollution connected fossil fuel–based power plants number of deaths due to air pollution

Energy Increased electricity generation from Solar installed capacity (MW); % solar of total
solar PV installed capacity; % solar of total installed
capacity of renewable energy sources

Access to clean, Increased access to clean, affordable and Number of houses/buildings/facilities with
affordable and reliable reliable energy access to clean energy resulting from the
energy policy

Capacity, skills Increase in training for skilled workers in Number of new skilled trainees and workers
and knowledge solar-relevant sectors on the ground
development

Jobs Increased jobs in the solar installation, Number of new jobs resulting from the policy
operations and maintenance sectors

Increased jobs in the solar panel Number of new jobs resulting from the policy
manufacturing sector

Decreased jobs in fossil fuel sectors Number of jobs reduced resulting from the
policy
178 Sustainable Development Methodology

TABLE A.1, continued

Impact categories, specific impacts and indicators included in the quantitative assessment boundary

Impact category Specific impacts Indicator to quantify

Income Increased income for households, Savings in annual electricity bill for
institutions and other organizations due households and businesses ($/year)
to reduction in energy costs

Energy independence Increased energy independence from Reduction in coal imports resulting from the
reduced imports of fossil fuel policy (t/year)

Chapter 8, Section 8.2: Choose assessment Section 8.3.2: Define the most likely baseline
method for each indicator scenario for each indicator
The first step is to choose an assessment method for
each indicator – the scenario method, comparison A key assumption about what is most likely to occur
group method or deemed estimates method (which in the absence of the solar PV policy is that the
is a subset of the scenario method); this is outlined households installing solar PV systems would have
in Section 8.2. In this example, the scenario method used grid-connected electricity in the absence of the
is used for certain indicators and the deemed solar PV policy.
estimates method for others. To apply the scenario
method, baseline values and policy scenario values Other policies
are needed for each indicator over the assessment The baseline scenario takes into account India’s
period. To apply the deemed estimates method, only National Solar Mission, which calls for 100,000 MW
the estimated change from the policy is quantified, of new solar capacity. Of the 100,000 MW of solar
without separately estimating baseline and policy power to be achieved by 2022, 40,000 MW is to
scenario values. be met by grid-connected rooftop solar systems
(included in the policy scenario), and the remaining
Chapter 8, Section 8.3: Define the baseline 60,000 MW is to be met by ground-based solar
scenario and estimate baseline values for each systems (included in the baseline scenario).
indicator
Section 8.3.1: Select a desired level of accuracy No other policies or subsidies are assumed to
and complexity exist for rooftop grid-connected solar PV systems.
No other financial incentives, such as soft loans
This example uses a combination of constant or capital grants for solar PV panels/systems, are
baseline scenarios and simple trend baseline assumed to be available.
scenarios for different indicators. Where the deemed
estimates method is used, no baseline values are The Government of India is also implementing
presented. the Off-Grid and Decentralized Solar Applications
scheme to promote solar home lights, solar street
A lower level of accuracy, commensurate with IPCC lights, power plants, solar pumps, and mini and
Tier 1 methods, was determined to be appropriate. micro grids in rural areas of the country, where a
For example, national-level data such as the national significant proportion of the population does not
average grid emission factor, country-wide rates of have access to electricity. The programme also has
solar PV as a percentage of total installed capacity, an emphasis on concentrating solar thermal (CST)
and national air pollution data can be considered technology. The objective and target user group
as representative within the impact category under the off-grid policy are different from those
assessment boundaries. of the solar PV incentive policy. Therefore, the off-
grid incentive policy has not been considered for
assessment.
 Appendices 179

Non-policy drivers Selected parameters included are listed in the

Table A.2 lists key drivers for each impact category Table A.3.
being assessed that is included in the baseline
scenario.

Section: 8.3.3: Define the methods and

parameters needed to estimate baseline values

Each indicator has its own estimation method and list

of parameters. These are shown in Table A.6.

TABLE A.2

Drivers and assumptions for the solar PV incentive policy

Impact category Drivers and assumptions in the baseline scenario

Climate change No change in emission limits from power plants and vehicles, and no change in compliance
mitigation rates

Health impacts of air No change in particulate matter limits from power plants, power generators or vehicles, and
pollution no change in compliance rates

Air pollution No change in air emission limits from power plants, power generators or vehicles, and no
change in compliance rates

Renewable energy No change in renewable energy targets, including the proportion of the target to be met by
generation solar

Access to clean, reliable No significant change in household income, production cost of solar systems, or number of
and affordable energy solar companies; no change in homeowners’ awareness of, and ability to invest in, solar PV
systems

Skilled labour and No change in access to, or awareness of, opportunities for solar PV industry training
worker training

Job creation No change in employment rate for skilled or unskilled labour

Income No significant change in average household income or inflation rate

Energy independence No change in the cost of fossil fuels or economic incentives for renewable energy
180 Sustainable Development Methodology

TABLE A.3

Parameters needed to estimate baseline values and data to be collected

Impact category Parameters and data

Climate change Grid electricity emission factor in India

mitigation Installed capacity of solar rooftop systems due to solar PV incentive policy

Air quality/health Emissions of PM2.5 and PM10 from stationary power plants, as reported by the Central Pollution
impacts of air Control Board, state pollution control boards and/or the National Environmental Engineering
pollution Research Institute
or
Reported levels of PM2.5 and PM10 in India (micrograms per cubic metre of air – μg/m3)
PM2.5 and PM10 that is attributable to power generation (%)

Air quality/health Emissions of SO2 and NOx from stationary power plants, as reported by the Central Pollution
impacts of air Control Board, state pollution control boards and/or the National Environmental Engineering
pollution Research Institute
or
Reported levels of SO2 and NOx in India
SO2 and NOx that are attributable to power generation (%)

Energy Total installed capacity of solar systems before implementation of the policy (MW)

Access to clean, Baseline values are not separately calculated because, within the assessment boundary, the
reliable and households that are assumed to adopt the policy already have access to energy and are simply
affordable energy replacing fossil sources with solar PV.

Capacity, skills Baseline values are not separately calculated because, within the assessment boundary, only the
and knowledge incremental increase in skilled labour associated with adoption of the policy is assessed.
development

Jobs Baseline values are not separately calculated because, within the assessment boundary, only the
incremental increase in job creation associated with adoption of the policy is being assessed.

Income Average expenditure on grid electricity

or
Average cost of grid-connected electricity consumed for residential and institutional use (Rs)

Energy Baseline values are not separately calculated because, within the assessment boundary, only the
independence incremental change in energy independence due to the policy is evaluated.

Section 8.3.4: Collect data for each indicator Chapter 9, Section 9.1: Define and describe the
policy scenario for each indicator
Data are collected for each parameter required for The following assumptions describe the policy
calculations. These are shown in Table A.6. scenario:

Section 8.3.5: Estimate baseline values for each • The policy is implemented in India over the
indicator period 2016–2022.

Baseline values are calculated over the assessment • The policy aims to install 40,000 MW of
period. These are shown in Table A.6. rooftop solar PV by 2022. Table A.4 shows the
 Appendices 181

TABLE A.4

Policy’s intended electricity generation over the assessment period

Year 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025

Installed rooftop 200 4,800 5,000 6,000 7,000 8,000 9,000 0 0 0

solar PV capacity
(MW)

Cumulative 200 5,000 10,000 16,000 23,000 31,000 40,000 40,000 40,000 40,000
installed rooftop
solar PV capacity
(MW)

Electricity 265.320 6,633 13,266 21,225.6 30,511.8 41,124.6 53,064 53,064 53,064 53,064
generation from
rooftop solar PV
(1,000 MWh/year)

annual and cumulative projected installed

capacity of solar PV systems in each year. The
table also shows the corresponding electricity
generated in each year from the solar PV. Each
MW of installed solar PV generates 1,327 MWh
of electricity per year.

Chapter 9, Section 9.2: Estimate policy scenario

values for each indicator
Policy scenario values are calculated over the
assessment period. These are shown in Table A.6.

Chapter 9, Section 9.3: Estimate the net impact

of the policy on each indicator
The net impact of the policy is calculated for each
indicator over the assessment period. These are
shown in Table A.6.

Table A.5 presents a summary of the net impact of

the policy across all impact categories included in the
quantitative assessment.
182 Sustainable Development Methodology

TABLE A.5

Summary of quantitative results for impact of solar PV incentive policy on all impact categories
included in the assessment

Estimated impact
Impact category Indicator quantified (cumulative, 2016–2025)

Climate change GHG emissions from the electricity grid Reduction of 307 MtCO2e
mitigation (MtCO2e)

Air quality/health PM2.5 emissions from the electricity grid (t) Reduction of 1,177,996 t PM2.5
impacts of air pollution
PM10 emissions from the electricity grid (t) Reduction of 2,437,234 t PM10

SO2 emissions from the electricity grid (t) Reduction of 4,265,161 t SO2

NOx emissions from the electricity grid (t) Reduction of 4,062,057 t NOx

Number of premature deaths per year in India Reduction of 32,304 premature

resulting from air pollution from coal plants deaths

Energy Renewable energy installed capacity (MW) Increase of 40,000 MW of renewable

energy capacity

Access to clean, Increase in number of houses/buildings/facilities Increase of 5,741,889 houses/

affordable and reliable with access to clean energy resulting from the buildings/facilities with access to
energy policy clean energy

Capacity, skills and Number of new skilled trainees and workers on Increase of 40,060 new skilled
knowledge development the ground because of the policy trainees and workers

Jobs Change in jobs resulting from the policy Net increase of 821,102 jobs
(number of jobs)

Income Savings in annual electricity bill for households Savings of $27,855 million
and businesses ($)

Energy independence Reduction in coal imports (t) Reduction of 57,770,140 t of coal

TABLE A.6

Calculations of baseline values, policy scenario values and the net impact of the policy on the indicators included in the assessment

Impact category 1 Climate change mitigation

Indicator GHG emissions from the electricity grid (MtCO2e/year)

Specific impact Reduced GHG emissions from grid-connected fossil fuel–based power plants

Assessment method Deemed estimates method

Equation GHG emissions reduced from the solar PV (MtCO2e/year) = electricity generated from rooftop solar PV (MWh) × coal generation emission factor
(tCO2e/MWh)/1,000,000

Parameters needed Electricity generated from new solar PV (MWh): see Table A.4
Coal generation emission factor = 0.945 tCO2e/MWh (for new coal power plants; emission factor assumed to stay constant over the assessment
period)

Assumptions It is assumed that, in the baseline scenario, new coal-based power plants will be added equivalent to the rooftop solar PV capacity addition due to
the proposed policy, and that no new diesel- and gas-based power plants will be added in future. Therefore, it is assumed that other fossil fuel–
based installed capacity (i.e. 9% of total grid, from diesel and gas) will not change in the baseline and policy scenarios.

Assessment Cumulative
period 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 impact

Reduction in GHG 0.25 6.27 12.54 20.06 28.83 38.86 50.15 50.15 50.15 50.15 307
emissions from the
policy (MtCO2e/year)
Appendices 183
TABLE A.6, continued

Calculations of baseline values, policy scenario values and the net impact of the policy on the indicators included in the assessment

Impact category 2 Air quality/health impacts of air pollution

Indicator 1 PM2.5 emissions from the electricity grid (t/year)

Specific impact Reduced PM2.5 emissions from grid-connected fossil fuel–based power plants

Assessment method Scenario method

Equation Reduction in PM2.5 emissions = baseline PM2.5 emissions – policy scenario PM2.5 emissions
184 Sustainable Development Methodology

where baseline PM2.5 emissions = total fossil fuel–based installed capacity of the grid (MW) in baseline scenario × PM2.5 emission factor (t/MW),
policy scenario PM2.5 emissions = total fossil fuel–based installed capacity of the grid (MW) in the policy scenario × PM2.5 emission factor (t/MW)

Parameters needed Installed capacity (MW) (see below) and PM2.5 emission factor = 4.8 t/MW per year

Assumptions It is assumed that, in the baseline scenario, new coal-based power plants will be added equivalent to the rooftop solar PV capacity addition
due to the proposed policy, and that no new diesel- and gas-based power plants will be added in future. Therefore, it is assumed that other
fossil fuel–based installed capacity (i.e. 9% of total grid, from diesel and gas) will not change in the baseline and policy scenarios.

Cumulative
Assessment period 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 impact

Baseline values: installed 184,274 197,976 211,677 225,379 239,081 252,783 266,485 260,571 247,422 250,106 -
capacity of coal-based
power plant (MW)

Policy scenario values: 184,074 192,976 201,677 209,379 216,081 221,783 226,485 220,571 207,422 210,106 -
installed capacity of coal-
based power plant (MW)

Baseline values: 885,293 951,120 1,016,947 1,082,774 1,148,600 1,214,427 1,280,254 1,251,841 1,188,671 1,201,568 -
PM2.5 emissions (t/year)

Policy scenario values: 884,332 927,099 968,904 1,005,906 1,038,103 1,065,496 1,088,085 1,059,672 996,502 1,009,399 -
PM2.5 emissions (t/year)

Reduction in PM2.5 961 24,021 48,042 76,868 110,497 148,931 192,169 192,169 192,169 192,169 1,177,996
emissions from the policy
(t/year)

Abbreviation: -, not applicable

TABLE A.6, continued

Calculations of baseline values, policy scenario values and the net impact of the policy on the indicators included in the assessment

Impact category 2 Air quality/health impacts of air pollution

Indicator 2 PM10 emissions from the electricity grid (t/year)

Specific impact Reduced PM10 emissions from grid-connected fossil fuel–based power plants

Assessment method Scenario method

Equation Reduction in PM10 emissions = baseline PM10 emissions – policy scenario PM10 emissions
where baseline PM10 emissions = total fossil fuel–based installed capacity of the grid (MW) in baseline scenario × PM10 emission factor (t/MW),
policy scenario PM10 emissions = total fossil fuel–based installed capacity of the grid (MW) in the policy scenario × PM10 emission factor (t/MW)

Parameters needed Installed capacity (MW) (see below) and PM10 emission factor = 9.9 t/MW per year

Assumptions It is assumed that, in the baseline scenario, new coal-based power plants will be added equivalent to the rooftop solar PV capacity addition
due to the proposed policy, and that no new diesel- and gas-based power plants will be added in future. Therefore, it is assumed that other
fossil fuel–based installed capacity (i.e. 9% of total grid, from diesel and gas) will not change in the baseline and policy scenarios.

Cumulative
Assessment period 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 impact

Baseline values: PM10 1,831,640 1,967,834 2,104,027 2,240,221 2,376,415 2,512,608 2,648,802 2,590,016 2,459,319 2,486,003 -
emissions (t/year)

Policy scenario values: PM10 1,829,652 1,918,135 2,004,630 2,081,185 2,147,800 2,204,475 2,251,211 2,192,425 2,061,728 2,088,412 -
emissions (t/year)

Reduction in PM10 emissions 1,988 49,699 99,398 159,037 228,615 308,133 397,591 397,591 397,591 397,591 2,437,234
from the policy (t/year)

Abbreviation: -, not applicable

Appendices 185
TABLE A.6, continued

Calculations of baseline values, policy scenario values and the net impact of the policy on the indicators included in the assessment

Impact category 2 Air quality/health impacts of air pollution

Indicator 3 SO2 emissions from the electricity grid (t/year)

Specific impact Reduced SO2 emissions from grid-connected fossil fuel–based power plants

Assessment method Scenario method

Equation Reduction in SO2 emissions = baseline SO2 emissions – policy scenario SO2 emissions
186 Sustainable Development Methodology

where baseline SO2 emissions = total fossil fuel–based installed capacity of the grid (MW) in baseline scenario × SO2 emission factor (t/MW),
project SO2 emissions = total fossil fuel–based installed capacity of the grid (MW) in the policy scenario × SO2 emission factor (t/MW)

Parameters needed Installed capacity (MW) (see below) and SO2 emission factor = 17.4 t/MW per year

Assumptions It is assumed that, in the baseline scenario, new coal-based power plants will be added equivalent to the rooftop solar PV capacity addition
due to the proposed policy, and that no new diesel- and gas-based power plants will be added in future. Therefore, it is assumed that other
fossil fuel–based installed capacity (i.e. 9% of total grid, from diesel and gas) will not change in the baseline and policy scenarios.

Cumulative
Assessment period 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 impact

Baseline values: 3,205,370 3,443,709 3,682,048 3,920,387 4,158,726 4,397,065 4,635,403 4,532,528 4,303,808 4,350,506 -
SO2 emissions (t/year)

Policy scenario values: 3,201,891 3,356,736 3,508,102 3,642,073 3,758,649 3,857,831 3,939,619 3,836,743 3,608,023 3,654,721 -
SO2 emissions (t/year)

Reduction in SO2 emissions 3,479 86,973 173,946 278,314 400,076 539,233 695,785 695,785 695,785 695,785 4,265,161
from the policy (t/year)

Abbreviation: -, not applicable

TABLE A.6, continued

Calculations of baseline values, policy scenario values and the net impact of the policy on the indicators included in the assessment

Impact category 2 Air quality/health impacts of air pollution

Indicator 4 NOx emissions from the electricity grid (t/year)

Specific impact Reduced NOx emissions from grid-connected fossil fuel–based power plants

Assessment method Scenario method

Equation Reduction in NOx emissions = baseline NOx emissions – policy scenario NOx emissions
where baseline NOx emissions = total fossil fuel–based installed capacity of the grid (MW) in baseline scenario × NOx emission factor (t/MW),
policy scenario NOx emissions = total fossil fuel–based installed capacity of the grid (MW) in the policy scenario × NOx emission factor (t/MW)

Parameters needed Installed capacity (MW) (see below) and NOx emission factor = 16.6 t/MW per year

Assumptions It is assumed that, in the baseline scenario, new coal-based power plants will be added equivalent to the rooftop solar PV capacity addition
due to the proposed policy, and no new diesel- and gas-based power plants will be added in future. Therefore, it is assumed that other fossil
fuel–based installed capacity (i.e. 9% of total grid, from diesel and gas) will not change in the baseline and policy scenarios.

Cumulative
Assessment period 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 impact

Baseline values: 3,052,734 3,279,723 3,506,712 3,733,702 3,960,691 4,187,681 4,414,670 4,316,693 4,098,865 4,143,339 -
NOx emissions (t/year)

Policy scenario values: 3,049,420 3,196,891 3,341,049 3,468,641 3,579,666 3,674,125 3,752,018 3,654,041 3,436,213 3,480,687 -
NOx emissions (t/year)

Reduction in NOx emissions 3,313 82,832 165,663 265,061 381,025 513,555 662,652 662,652 662,652 662,652 4,062,057
from the policy (t/year)

Abbreviation: -, not applicable

Appendices 187
TABLE A.6, continued

Calculations of baseline values, policy scenario values and the net impact of the policy on the indicators included in the assessment

Impact category 2 Air quality/health impacts of air pollution

Indicator 5 Number of premature deaths per year in India resulting from air pollution from coal plants

Specific impact Reduction in premature mortality in India from reduced fossil fuel electricity generation

Assessment method Scenario method

Equation Reduction in premature deaths per year = expected premature deaths in baseline scenario – expected premature deaths in policy scenario
188 Sustainable Development Methodology

Parameters needed Installed capacity (MW) (see below) and premature deaths = 0.81/MW installed capacity per year

Assumptions It is assumed that, in the baseline scenario, new coal-based power plants will be added equivalent to the rooftop solar PV capacity addition
due to the proposed policy, and that no new diesel- and gas-based power plants will be added in future. Therefore, it is assumed that other
fossil fuel–based installed capacity (i.e. 9% of total grid, from diesel and gas) will not change in the baseline and policy scenarios.
The total health risk for mortality is quantified using the relative risk functions and exposure level for PM2.5. The premature deaths per MW
applied for this example are based on previously published literature and are extrapolated for simplification.

Cumulative
Assessment period 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 impact

Baseline values (cumulative) 148,821 159,886 170,952 182,018 193,084 204,149 215,215 210,439 199,820 201,988 -

Policy scenario values 148,659 155,848 162,876 169,096 174,509 179,114 182,911 178,135 167,515 169,683 -
(cumulative)

Reduction in premature 162 4,038 8,076 12,922 18,575 25,036 32,304 32,304 32,304 32,304 32,304
deaths (cumulative)

Abbreviation: -, not applicable

TABLE A.6, continued

Calculations of baseline values, policy scenario values and the net impact of the policy on the indicators included in the assessment

Impact category 3 Energy

Indicator Renewable energy installed capacity (MW)

Specific impact Increased renewable energy generation from more solar generation

Assessment method Scenario method

Equation Total renewable energy installed capacity (MW) = renewable energy capacity in baseline scenario – renewable energy capacity in policy
scenario

Parameters needed Baseline values of total renewable energy without the policy (MW)
Policy scenario values of total renewable energy with the policy each year (MW)

Assumptions See Table A.4

Cumulative
Assessment period 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 impact

Baseline values: total 42,649 54,674 72,739 89,804 105,870 120,935 135,000 139,613 144,226 148,839 -
renewable energy without
the policy (MW) (cumulative)

Policy scenario values: total 42,849 59,674 82,739 105,804 128,870 151,935 175,000 179,613 184,226 188,839 -
renewable energy with the
policy (MW) (cumulative)

Increase in renewable 200 5,000 10,000 16,000 23,000 31,000 40,000 40,000 40,000 40,000 40,000
energy capacity (MW)
(cumulative)

Percentage increase in 0 9 14 18 22 26 30 29 28 27 -
renewable energy capacity
(%)

Abbreviation: -, not applicable

Appendices 189
TABLE A.6, continued

Calculations of baseline values, policy scenario values and the net impact of the policy on the indicators included in the assessment

Impact category 4 Access to clean, affordable and reliable energy

Indicator Increase in number of houses/buildings/facilities with access to clean energy resulting from the policy

Specific impact Increased access to clean electricity

Assessment method Deemed estimates method

Equation Number of installations = total installed capacity target in eligible sector (i.e. residential, institutional, industrial, commercial and government)/
standard solar rooftop installation size for each type of installation/1,000
190 Sustainable Development Methodology

Parameters needed Standard solar rooftop system size for each type of installation (kW)
Total installed capacity target in eligible sector (i.e. residential, institutional, industrial, commercial and government) (MW)

Assumptions The solar PV incentive policy sets target for eligible sectors. Total new installations are estimated using a standard size and target of the
eligible category.

Cumulative
Assessment period 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 impact

Residential 24,000 576,000 600,000 720,000 840,000 960,000 1,080,000 0 0 0 4,800,000

(number of households)

Institutional 240 5,760 6,000 7,200 8,400 9,600 10,800 0 0 0 48,000

(number of buildings)

Industrial 3,375 81,000 84,375 101,250 118,125 135,000 151,875 0 0 0 675,000

(number of facilities)

Commercial 1,050 25,200 26,250 31,500 36,750 42,000 47,250 0 0 0 210,000

(number of buildings)

Government 44 1,067 1,111 1,333 1,556 1,778 2,000 0 0 0 8,889

(number of buildings)

Increase in number of 28,709 689,027 717,736 861,283 1,004,831 1,148,378 1,291,925 0 0 0 5,741,889
houses/buildings/facilities
with access to clean energy
resulting from the policy
(houses/buildings)
TABLE A.6, continued

Calculations of baseline values, policy scenario values and the net impact of the policy on the indicators included in the assessment

Impact category 5 Capacity, skills and knowledge development

Indicator Number of new skilled trainees and workers on the ground because of the policy per year

Specific impact Increase in training for skilled workers in solar-relevant sectors

Assessment method Deemed estimates method

Equation Target for new skilled trainees and workers on the ground per year

Parameters needed Target for new skilled trainees and workers on the ground per year

Assumptions The solar PV incentive policy includes targets to train new workers to support the policy goals.

Cumulative
Assessment period 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 impact

Number of new skilled 460 5200 6000 8400 8000 8000 4000 0 0 0 40,060
trainees and workers on
the ground because of the
policy per year
Appendices 191
TABLE A.6, continued

Calculations of baseline values, policy scenario values and the net impact of the policy on the indicators included in the assessment

Impact category 6 Jobs

Indicator Change in jobs resulting from the policy (jobs/year)

Specific impact Increased jobs in the solar panel manufacturing, construction and installation, and operation and maintenance sectors
Reduced jobs in fossil fuel sectors

Assessment method Deemed estimates method

Equation Total jobs = total capacity (MW) × jobs per MW

192 Sustainable Development Methodology

Parameters needed Jobs per MW = manufacturing (11 jobs/MW, of which 40% are domestic); installation (13 jobs/MW); operation and maintenance (3.5 jobs/MW);
fossil fuel sector (1 job/MW)
Installed capacity (MW)

Assumptions It is assumed that 70% of planned capacity will likely come from new fossil fuel–based power plants.

Cumulative
Assessment period 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 impact

Solar panel manufacturing 879 21,097 21,976 26,371 30,766 35,162 39,557 0 0 0 175,808

Construction and 2,640 63,360 66,000 79,200 92,400 105,600 118,800 0 0 0 528,000
installation

Operation and maintenance 702 16,848 17,550 21,060 24,570 28,080 31,590 0 0 0 140,400

Fossil fuel sector –139 –3,143 –3,103 –3,555 –3,984 –4,393 –4,789 0 0 0 –23,106

Net change in jobs 4,082 98,162 102,423 123,076 143,753 164,448 185,158 0 0 0 821,102
(jobs/year)
TABLE A.6, continued

Calculations of baseline values, policy scenario values and the net impact of the policy on the indicators included in the assessment

Impact category 7 Income

Indicator Savings in annual electricity bill for households and businesses ($/year)

Specific impact Increased income for households, institutions and other organizations due to reduction in energy costs

Assessment method Deemed estimates method

Equation Savings on electricity bill = total electricity generated from solar rooftop by sector (kWh) × tariff by sector ($/kWh)

Parameters needed Total units generated (kWh) (see Table A.4)

Tariff: household and institutional ($0.08/kWh); commercial ($0.12/kWh)

Assumptions The annual escalation in tariff is assumed to be 4%.

Cumulative
Assessment period 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 impact

National reduction in 27 566 1,178 1,960 2,930 4,107 5,512 4,586 3,815 3,174 27,855
electric bills (million $/year)
Appendices 193
TABLE A.6, continued

Calculations of baseline values, policy scenario values and the net impact of the policy on the indicators included in the assessment

Impact category 8 Energy independence

Indicator Reduction in coal imports (t/year)

Specific impact Increased energy independence from reduced imports of coal

Assessment method Deemed estimates method

Equation Reduction in coal imports = electricity generated from new solar PV (MWh) × coal consumption per unit of electricity (t/MWh) × coal import
ratio (%)
194 Sustainable Development Methodology

Parameters needed Electricity generated from new solar PV (MWh/year) (see Table A.4)
Coal consumption per unit of electricity (t/MWh) = 0.74 t/MWh
Coal import ratio (%) = 24%

Assumptions It is assumed that, in the baseline scenario, new coal-based power plants will be added equivalent to the rooftop solar PV capacity addition
due to the proposed policy, and that no new diesel- and gas-based power plants will be added in future. It is also assumed that the coal
reduction will have a proportional impact on imports and domestic coal. It is further assumed that coal efficiency and the coal import ratio will
stay the same for the next 10 years.

Cumulative
Assessment period 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 impact

Reduction in coal imports 47,121 1,178,021 2,356,042 3,769,667 5,418,896 7,303,729 9,424,166 9,424,166 9,424,166 9,424,166 57,770,140
from the policy (t/year)
Appendix B: Stakeholder participation
during the assessment process

This appendix provides an overview of the ways steps in the assessment process where stakeholder
that stakeholder participation can enhance the participation is recommended and why it is
sustainable development impact assessment process important, noting where relevant guidance can be
and the contribution of policies to sustainable found in the ICAT Stakeholder Participation Guide.
development. Table B.1 provides a summary of the

TABLE B.1

Steps where stakeholder participation is recommended in the impact assessment

Step of sustainable Relevant chapters in

development impact Why stakeholder participation is important at Stakeholder Participation
assessment this step Guide

Chapter 2 – Objectives • Ensure that the objectives of the assessment Chapter 5 – Identifying and
of assessing sustainable respond to the needs and interests of stakeholders understanding stakeholders
development impacts

Chapter 3 – Key • Build understanding, participation and support for Chapter 4 – Planning effective
concepts, steps the policy among stakeholders stakeholder participation
and planning the • Ensure conformity with national and international Chapter 5 – Identifying and
assessment laws and norms, as well as donor requirements understanding stakeholders
• Section 3.3 – Planning related to stakeholder participation Chapter 6 – Establishing multi-
the assessment • Identify and plan how to engage stakeholder groups stakeholder bodies
who may be affected or may influence the policy Chapter 9 – Establishing
• Coordinate participation at multiple steps of this grievance redress mechanisms
assessment with participation in other stages of the
policy design and implementation cycle, and other
assessments

Chapter 5 – Choosing • Enhance completeness by including impact Chapter 5 – Identifying and

which impact categories categories that are relevant and significant for understanding stakeholders
and indicators to assess the priorities and concerns of diverse stakeholder Chapter 7 – Providing
groups information to stakeholders
• Identify and address possible unintended or Chapter 8 – Designing and
negative impacts early on conducting consultations
• Identify credible sources of information for selected
indicators

Chapter 6 – Identifying • Strengthen identification and assessment of Chapter 8 – Designing and

specific impacts within sustainable development impacts conducting consultations
each impact category • Enhance completeness by identifying impacts for
different stakeholder groups
• Integrate stakeholder insights about cause–effect
relationships between the policy and impacts
• Identify and address possible unintended or
negative impacts
196 Sustainable Development Methodology

TABLE B.1, continued

Steps where stakeholder participation is recommended in the impact assessment

Step of sustainable Relevant chapters in

development impact Why stakeholder participation is important at Stakeholder Participation
assessment this step Guide

Chapter 7 – Qualitatively • Ensure that the assessment period responds to Chapter 8 – Designing and
assessing impacts stakeholders’ needs conducting consultations
• Gain insights into a policy’s specific local context and
impacts
• Strengthen evidence base of the assessment
• Integrate stakeholder insights on likelihood and
magnitude of impacts, and the nature of change

Chapter 12 – Monitoring • Ensure relevance and completeness of indicators to Chapter 8 – Designing and
performance over time be monitored conducting consultations
• Ensure that monitoring frequency addresses the
needs of decision makers and other stakeholders
• Assess impacts on different stakeholder groups to
identify and manage trade-offs

Chapter 13 – Reporting • Raise awareness of benefits and other impacts to Chapter 7 – Providing
build support for the policy information to stakeholders
• Ensure that reports and summaries properly
characterize the impacts for each category
• Inform decision makers and other stakeholders
about impacts, including differentiated impacts
on different stakeholder groups, to allow adaptive
management to reduce negative and enhance
positive impacts
• Increase accountability and transparency,
and thereby credibility and acceptance of the
assessment

Chapter 14 – Evaluating • Ensure that diverse perspectives are considered Chapter 7 – Providing
synergies and when doing a CEA, CBA or MCA, especially regarding information to stakeholders
trade-offs, and using subjective elements such as valuation of social Chapter 8 – Designing and
results and environmental benefits, and weighting the conducting consultations
importance of different impacts
• Ensure that diverse perspectives are considered,
especially those of affected communities, when
making decisions about whether to continue or
discontinue policies, make changes to policies, or
implement new policies
Appendix C: Qualitative research methods

Qualitative methods can be flexible. They may Qualitative methods (especially story-based
involve several methods and approaches, such as approaches) can yield powerful stories, which can
stakeholder interviews, surveys, focus groups, case be useful for media reports, and are often preferred
studies, literature review and direct observations, by policymakers and politicians. Hard data are
using narrative descriptions. not always the most convincing evidence for all
audiences.
Interviews and case studies are useful to gain
insights into a policy’s specific local context and The approach used will depend on the goals of
impacts, as well as the attitudes, experiences the assessment. To determine which type of data
and perspectives of affected stakeholders and to collect, users need to determine what is most
participants. On the other hand, they tend to be important to the policy under assessment. Is the
limited in coverage and therefore non-representative goal to collect numerical data on the use of solar PV
of broader conditions or impacts, which can produce or provide a more in-depth understanding of the
less reliable results with less ability to generalize and situation in the poorest urban areas? Sometimes
quantify impacts. Therefore, it can be helpful to use a both approaches are important, but resource
combination of qualitative and quantitative data and availability may require that one must be given
approaches. priority.

Quantitative approaches should be used if a user

wants to conduct numerical or statistical analysis,
wants to be precise, knows what can be measured, C.1 Forms of data collection
or wants to cover a large group. On the other hand,
qualitative approaches should be used if a user Data-collection approaches can be considered
wants narrative or in-depth information, is not sure structured or semi-structured. A structured
what can be measured, or does not need to quantify data-collection approach requires that all data
the results.66 be collected in exactly the same way. Structured
data collection allows users to compare findings
Qualitative methods are used specifically to consider at different sites to draw conclusions about what
the “why” questions that quantitative methods is working where. A structured approach is also
typically cannot answer: important when comparing alternative interventions
to determine which is most cost-effective. Structured
• Why does the policy work (or not work)? data collection is mostly used to collect quantitative
data when the user has a large sample or population,
• How does the policy achieve its goals? knows what needs to be measured, needs to show
results numerically, or needs to make comparisons
• Why does the policy work in some situations across different sites or interventions.
and not others?
A semi-structured data-collection approach may be
• What needs of the population are/were not systematic and follow general procedures, but data
anticipated? are not collected in the same way every time. Semi-
structured interviews, for example, are often based
• What were the additional unintended on a predetermined set of broad questions, but the
and/or unexpected positive or negative order of presenting the questions may depend on
consequences? circumstances. Moreover, some responses provided
can be probed with additional questions developed
during the interview. This approach is more open
and fluid than the structured approach. The semi-
structured approach allows respondents to tell users
66
Morra Imas and Rist (2009). what they want to know in their own way.
198 Sustainable Development Methodology

Semi-structured data-collection methods are Data can also be collected obtrusively or

generally qualitative. They are used when a user is unobtrusively. Obtrusive methods are observations
conducting exploratory work in a new development made with participants’ knowledge. Such methods
area, seeks to understand themes or issues, or are used to measure perceptions, opinions and
wants participant narratives or in-depth information. attitudes through interviews, surveys and focus
They can also be used to understand results of groups. Observations made with the knowledge
structured data collection that are unexpected and of those being observed are also obtrusive.
not well understood, or to give nuanced examples Unobtrusive methods are observations made
to supplement the findings from a structured data- without the knowledge of the participant. Examples
collection effort. of unobtrusive methods are using data from
documents or archives, and observing participants
For example, in an evaluation of a community-driven without their knowledge.
development project, evaluators might choose a
semi-structured approach to data collection. Because Data collection usually includes both quantitative and
such programmes give control of planning decisions qualitative data, but one approach may be dominant.
to local groups, it is appropriate for the evaluator The two approaches can be characterized as shown
to use a semi-structured approach to learn more in Table C.1.
about how decisions are made, as well as to solicit
community members’ views of the process and Box C.1 provides a checklist to help decide which
project outcomes. data-collection approaches are most appropriate.

TABLE C.1

Summary of quantitative and qualitative approaches

Quantitative approach Qualitative approach

More structured Less structured

Emphasizes reliability Easier to develop
Harder to develop Can provide nuanced data (idiosyncratic data on each unit being studied)
Easier to analyse More labour-intensive to collect and analyse data
Emphasizes validity

Source: Morra Imas and Rist (2009).

 Appendices 199

BOX C.1

20-question qualitative checklist

1. Does the programme emphasize individual outcomes – that is, are different participants expected to be affected in
qualitatively different ways? Is there a need or desire to describe and evaluate these individual client outcomes?
2. Are decision makers interested in elucidating and understanding the internal dynamics of programmes – programme
strengths, programme weaknesses and overall programme processes?
3. Is detailed, in-depth information needed about certain client cases or programme sites (e.g. particularly successful cases,
unusual failures, critically important cases) for programmatic, financial or political reasons?
4. Is there interest in focusing on the diversity among, idiosyncrasies of, and unique qualities exhibited by, individual clients
and programmes (as opposed to comparing all clients or programmes on standardized, uniform measures)?
5. Is information needed about the details of programme implementation – that is, what do clients in the programme
experience? What services are provided to clients? How is the programme organized? What do staff members do?
Do decision makers need to know what is going on in the programme and how it has developed?
6. Are the programme staff and other stakeholders interested in collection of detailed, descriptive information about the
programme for the purpose of improving the programme (i.e. is there interest in formative evaluation)?
7. Is there a need for information about the nuances of programme quality – descriptive information about the quality of
programme activities and outcomes, not just levels, amounts or quantities of programme activity and outcomes?
8. Does the programme need a case-specific quality assurance system?
9. Are legislators or other decision makers or funders interested in having evaluators conduct programme site visits so
that the evaluations can be the surrogate “eyes and ears” for decision makers who are too busy to make such site visits
themselves, and who lack the observing and listening skills of trained evaluators? Is legislative monitoring needed on a
case-by-case basis?
10. Is the obtrusiveness of evaluation a concern? Will the administration of standardized measuring instruments
(questionnaires and tests) be overly obtrusive, in contrast to data gathering through natural observations and open-
ended interviews? Will the collection of qualitative data generate less reactivity among participants than the collection of
quantitative data? Is there a need for unobtrusive observations?
11. Is there a need and desire to personalize the evaluation process by using research methods that emphasize personal,
face-to-face contact with the programme – that is, methods that may be perceived as “humanistic” and personal because
they do not label and number the participants, and they feel natural, informal and understandable to participants?
12. Is a responsive evaluation approach appropriate – that is, an approach that is especially sensitive to collecting descriptive
data and reporting information in terms of differing stakeholder perspectives, based on direct, personal contact with
these stakeholders?
13. Are the goals of the programme vague, general and non-specific, indicating the possible advantage of a goal-free
evaluation approach that would gather information about what effects the programme is actually having rather than
measure goal attainment?
14. Is there a possibility that the programme may be affecting clients or participants in unanticipated ways and/or having
unexpected side effects, indicating the need for a method of inquiry that can discover effects beyond those formally
stated as desirable by programme staff (again, an indication of the need for some form of goal-free evaluation)?
15. Is there a lack of proven quantitative instrumentation for important programme outcomes? Is the state of measurement
science such that no valid, reliable and believable standardized instrument is available, or can be readily developed, to
measure quantitatively the particular programme outcomes for which data are needed?
16. Is the evaluation exploratory? Is the programme at a pre-evaluation stage, where goals and programme content are still
being developed?
17. Is an evaluability assessment needed to determine a summative evaluation design?
18. Is there a need to add depth, detail and meaning to statistical findings or survey generalizations?
19. Has the collection of quantitative evaluation data become so routine that no one pays much attention to the results
anymore, suggesting a possible need to break the old routine and use new methods to generate new insights about the
programme?
20. Is there a need to develop a programme theory grounded in observations of programme activities and impacts, and the
relationship between treatment and outcomes?

Source: Patton (1987).

200 Sustainable Development Methodology

To collect data on a policy, it is important to apply Although samples can be smaller, it is still vital to
rules in the data-collection process. Some of the ensure that the sample resembles the whole group
data-collection rules are in Box C.2. as closely as possible. Therefore, users should:

• have a clear idea of the characteristics of the

group they are assessing
C.2 Sampling in qualitative impact
assessment • create a sample that attempts to reflect the
range of different people in the group; for
Qualitative impact assessment involves engaging example, if the policy affects equal numbers
with people and talking to them. This can be of women and men, the qualitative sample
time-consuming and generate a large amount of should contain equal numbers of women
data to analyse. For example, policies are likely to and men.
affect thousands of people; setting up interviews
and analysing transcripts for each of them will be A particularly important goal of sampling in
expensive and may divert the user from other tasks. qualitative impact assessment is involving people
Sampling systematically enables the user to select a who have been less engaged in the policy and those
representative smaller group of participants from the who do not volunteer themselves to be consulted.
overall population who can give a reliable account of If the user only collects information from those who
the bigger picture. have been affected by the policy or are the first to
volunteer, the sampling will not be representative of
The way users select the sample has implications the population as a whole, and the assessment will
for the conclusions users can draw. Sampling not be credible.
for qualitative impact assessment has a slightly
different aim from sampling in quantitative impact
assessment. In quantitative impact assessment,
the goal is to draw a sample that is mathematically C.3 Longitudinal impact assessment
representative of the whole, so can be used to draw
firm conclusions about the population. In qualitative To show change over time, it is useful to speak to
impact assessment, precise or definitive conclusions the same people at multiple points in time to see
are less important, so sample sizes can be smaller – how their experiences have changed, rather than
the goal is to learn about the range of experiences of collecting information only once. Longitudinal
stakeholders. qualitative impact assessment provides nuanced
information on people’s perspectives, and how and
why they have changed over time, which can give a
fuller assessment of policy impact.

BOX C.2

Rules for collecting data

Evaluators should apply the following rules in collecting data:

• Use multiple data-collection methods when possible.
• Use available data if possible (doing so is faster, less expensive and easier than generating new data).
• If using available data, find out how earlier evaluators collected the data, defined the variables and ensured accuracy of
the data. Check the extent of missing data.
• If original data must be collected, establish procedures and follow them (protocol), maintain accurate records of
definitions and coding, pre-test, and verify the accuracy of coding and data input.
• Collect data in a disaggregated manner, to understand whether there are differences in views, impacts and economic
opportunities between women and men, people with different ethnicities, and other groups.

Source: Adapted from Morra Imas and Rist (2009).

 Appendices 201

C.4 Avoiding bias • participatory methods

The data-collection technique chosen will depend • ethnography

on the situation. Whichever method is chosen
to gather data from people, all the information • documents and other sources
gathered is potentially subject to bias. One
form of bias results from the fact that, when • case study approaches.
asked to provide information about themselves
or others, respondents may not tell the whole
truth, unintentionally or intentionally. They may C.5.1 Surveys
distort the truth because they do not remember
accurately or fear the consequences of providing a Surveys can be excellent tools for collecting data
truthful answer. They may also be embarrassed or about people’s perceptions, opinions and ideas. They
uncomfortable about admitting things they feel will are less useful in measuring behaviour, because
not be socially acceptable. All self-reported data are what people say they do may not reflect what they
vulnerable to this problem. actually do. Surveys can be structured or semi-
structured, administered in person or by telephone,
Selection bias may also exist. Selection bias occurs or self-administered by having people respond to a
when the people who choose to participate in the mailed or web form. Surveys can poll a sample of the
survey are different from those who choose not population or all of the population. There are two
to participate. This is often a challenge in surveys, types of surveys:
interviews and focus groups. Those who volunteer
to participate may be systematically different from • Structured surveys are surveys that include
those who do not. a range of response choices, one or more
of which are selected by respondents. All
respondents are asked exactly the same
questions in exactly the same way and given
C.5 Tools for collecting data exactly the same answers to choose from.

Typically, more than one data-collection approach • Semi-structured surveys are surveys that ask
is used to answer different impact assessment predominantly open-ended questions. They
questions or provide multiple sources of data in are especially useful when the user wants to
response to a single impact assessment question. gain a deeper understanding of reactions to
Users may, for example, collect available data for experiences or to understand the reasons
solar PV installation records, interview buyers on why respondents hold particular attitudes.
the use of solar PV, and survey users. Sometimes Semi-structured surveys should have a clearly
investigators use focus groups or conduct case defined purpose. It is often more practical to
studies to help develop themes for a questionnaire interview people about the steps in a process,
or to make sense of survey results. the roles and responsibilities of various
members of a community or team, or how a
Collecting the same information using different programme works than to attempt to develop
methods to increase the accuracy of the data is a written survey that captures all possible
called a triangulation of methods. Evaluators use variations.
triangulation to strengthen findings. The more
information gathered using different methods that Box C.3 gives example of questions in structured and
supports a finding, the stronger the evidence is. semi-structured surveys.

The following data-collection tools can be used, When conducting surveys, it is important to ensure
depending on which are most appropriate for a given representative samples to draw meaningful
situation: conclusions about the broader population of interest
and avoid selection bias. Obtaining a credible and
• surveys representative response from the population of
interest can sometimes be time-consuming and
• interviews expensive.

• focus groups
202 Sustainable Development Methodology

BOX C.3

Structured and semi-structured survey questions

Examples of structured questions:

1. Has this workshop been useful in helping you to learn how to evaluate your programme?
• Little or no extent
• Some extent
• Moderate extent
• Great extent
• Very great extent
• No opinion
• Not applicable
2. Do all people in the village have a source of clean water within 500 metres of their homes?
• Yes
• No
Examples of semi-structured questions:
• What have you learned from the programme evaluation workshop that you have used on the job?
• Where are the sources for clean water for the villagers?

Source: Morra Imas and Rist (2009).

In qualitative assessment impact, interview

C.5.2 Interviews questions should have the following
characteristics:
One of the most common methods of collecting
qualitative data is interviewing people – that is, • Open ended to encourage full responses.
talking to them one-to-one. Interviews can be Minimize yes/no questions; instead, try to
undertaken in person, by phone or over the internet start questions with “how”, “what”, “why”
(e.g. using Skype). Table C.2 describes three different and “where” to encourage interviewees to
approaches to interviewing. explore their answers.

Of the options in Table C.2, semi-structured • Clear and in plain English. Avoid long
interviewing is often the most promising approach or complex questions. Instead of asking
for carrying out qualitative impact assessment. The “What was the impact of …”, try “Did
approach allows the user to guide the direction anything change after …”.
and themes of the interview, while still allowing the
respondent to articulate their experiences in detail. • Framing rather than leading. Do not
point interviewees towards a particular
Another valuable approach is to combine structured response. Instead of “Did you feel better
“tick box” type questions with more open-ended after …”, ask “How did you feel after …”.
questions within the same interview. This provides
both numerical impact results and more nuanced • Neutral. Using emotive language or
qualitative information. asking in a way that sounds accusatory
may close down people’s responses.
Instead of “Did you do ...”, ask “How many
times have you done …” to imply that
others also do so.

Source: Morra Imas and Rist (2009).

 Appendices 203

TABLE C.2

Interview approaches

Structured Semi-structured Unstructured

Description Questions are agreed in The main questions are fixed, Interviewer may have a list
advance; interviewers stick but follow-up questions can of broad topics, but no set
rigidly to a script. be improvised. questions.

When to use Useful for collecting Most common in qualitative More appropriate for very
standardized, survey-style work; allows expanded exploratory research questions
information. opinions on the topics of the or academic research;
interview. direction is set by the
interviewee, rather than the
interviewer, so topics vary.

Sampling Sample sizes can be large, Longer interviews require Longer interviews require
and and time commitment is more time, so it is more more time, so it is more suited
minimal. suited to smaller samples to smaller samples targeting
Random sampling is targeting particular particular participants.
recommended for maximum participants.
rigour.

Transcribing Easy because all responses Mixed Time-consuming; full

are on the same template. transcription or detailed notes
and recording may be needed.

Data Easy to compare and analyse, Mixed Difficult to analyse, but provide
analysis but detail and nuance limited. detailed and nuanced data.

Source: Adapted from Arksey and Knight (1999).

C.5.3 Focus groups Group interviews provide group data, since

participants play off against each other. This can be
Focus groups are interviews with small groups of positive, allowing ideas to develop and be discussed
people. Numbers should be restricted to around in detail. However, it is important for the user to note
6–8 participants to prevent subgroups emerging and that an individual’s response in a focus group cannot
to make transcribing easier. In some cases, mini- be considered in the same way as an individual
groups of three or four may be most suitable. interview. Participants influence each other, and
responses should be seen in that context. When
Focus groups may be useful: analysing focus group data, avoid talking about
magnitude. For example, three out of six participants
• where time is too limited to conduct individual making a statement does not necessarily mean that
interviews 50% of participants agree with it, particularly because
they can be influenced by each other.
• for a collective discussion among a similar
or differing group, since the group dynamics Focus groups can have disadvantages. They can
can encourage more lively and interesting be hard to set up and organize, and difficult to
discussions moderate. They are not good for discussing sensitive
or personal topics. Unless the user has skills in
• where participants do not feel confident about drawing out quieter members of the group, the
taking part in individual interviews. views can be strongly influenced by the most vocal or
dominant participants.
204 Sustainable Development Methodology

C.5.4 Participatory methods C.5.7 Case study approaches

Impact assessment is participatory when the Case studies are widely used in impact assessment.
population under study is actively involved in They are not a method of data collection in
designing the assessment or collecting data. For themselves, but rather an approach that focuses on
example, participatory methods have been used gathering a range of evidence about a small number
in international development projects to give local of cases. They show the policy impact in a balanced
people a say in how projects are run, and to use way. Case studies should be chosen systematically,
local knowledge to better tailor the project and its as would be done for a sample for interviews or
measurement to specific contexts. surveys. In particular, it is important to capture a
wide spectrum of experiences of the policy, not just
Participatory methods can be used to collect the cases in which the project worked best.
qualitative evidence of impact. Project participants
gather data using methods such as photography or To create credible case studies, users should
video, giving a highly personal account of their own choose a small sample of cases randomly or based
lives and experiences. Other participatory methods on certain criteria. Users can use the methods
include creating diaries or “route-maps” with users, in described above to gather more information about
which they plot events on a timeline. These methods each selected case (e.g. interviews, focus groups,
can help to highlight the link between certain life observation, quantitative data, documents relating to
events and levels of engagement with a project, the case). The aim is to create a nuanced description
giving a sense of external influences. of how a policy has (or has not) affected individuals
and the reasons for change, as well as any other
Participatory methods can give nuanced information factors that are important.
on the effects of a policy, but are resource-intensive.
They also lack objectivity and any method of
comparing impacts on different individuals. C.5.8 Using multiple methods

In general, many of the above techniques

C.5.5 Ethnography for collecting data can be used. In qualitative
assessments, partly as a quality control mechanism,
Ethnography involves observing things from the the use of multiple methods (also called “mixed
point of view of those being studied. Rather than methods”, especially when quantitative methods
talking to people about their experiences, the are included) is common. It also yields more robust
ethnographer joins in and sees it first-hand. For results on the basis of triangulation – that is, use
example, it may be used to help understand how of different methods, with different sources of
people are engaging with community services staff. data and from different perspectives, to gain the
best understanding and produce the most credible
results.
C.5.6 Documents and other sources

Although qualitative data collected face-to-face are

ideal, in some cases users may not need to collect
data directly. Instead, the required information
may be found in existing documents. For example,
some qualitative data may be available from open-
ended questions in a quantitative survey or from key
workers’ case notes. Media articles about a particular
topic can also be useful, or users may want to
analyse local strategy documents to show variation in
attitudes or services.

Although these data are already available,

collecting and analysing the data systematically is
still important. It will help to show that users have
included data from all participants or a systematically
selected sample, and that users have completed a
thorough search for publicly available material.
Abbreviations and acronyms

CBA cost–benefit analysis NOX nitrogen oxides

CEA cost-effectiveness analysis PM particulate matter

CO2e carbon dioxide equivalent PV photovoltaic

DTU Technical University of Denmark Rs Indian rupee

FAO Food and Agriculture Organization SDG Sustainable Development Goal

of the United Nations
SO2 sulfur dioxide
GDP gross domestic product
t tonne
GHG greenhouse gas
TWG Technical Working Group
GW gigawatt
UNEP United Nations Environment
ha hectare Programme

ICAT Initiative for Climate Action UNFCCC United Nations Framework

Transparency Convention on Climate Change

IPCC Intergovernmental Panel on

Climate Change

kg kilogram

km kilometre

kW kilowatt

kWh kilowatt-hour

m3 cubic metre

MRV monitoring, reporting and

verification

MCA multi-criteria analysis

Mt megatonne

MW megawatt

MWh megawatt-hour

NDC nationally determined contribution

NGO non-governmental organization

Glossary

Adopted policies Policies for which an official government decision has been made and there is a
clear commitment to proceed with implementation, but implementation has not
yet begun

Assessment boundary The scope of the assessment in terms of the range of dimensions, impact
categories and specific impacts that are included in the assessment

Assessment period The time period over which impacts resulting from a policy are assessed

Assessment report A report, completed by the user, that documents the assessment process, and the
GHG, sustainable development and transformational impacts of a policy

Baseline scenario A reference case that represents the events or conditions most likely to occur in
the absence of a policy (or package of policies) being assessed

Baseline value The value of a parameter in the baseline scenario

Bottom-up data Data that are measured, monitored or collected at the facility, entity or project level

Causal chain A conceptual diagram tracing the process by which a policy leads to impacts
through a series of interlinked logical and sequential stages of cause-and-effect
relationships

Dimension An overarching category of sustainable development impacts. There are three

dimensions: environmental, social and economic.

Drivers Socioeconomic or other conditions, or other policies that affect an impact category.
For example, economic growth is a driver of increased energy consumption.
Drivers are divided into two types: other policies and non-policy drivers.

Dynamic A descriptor for a parameter that changes over time

Ex-ante assessment The process of assessing expected future impacts of a policy (i.e. a forward-looking
assessment)

Ex-ante baseline scenario A forward-looking baseline scenario, based on forecasts of external drivers (such
as projected changes in population, economic activity or other drivers that affect
emissions), in addition to historical data

Expert judgment A carefully considered, well-documented qualitative or quantitative judgment

made in the absence of unequivocal observational evidence by a person or
persons who have a demonstrable expertise in the given field.67 Users can
apply their own expert judgment or consult experts. Expert judgment can be
strengthened through expert elicitation methods to avoid bias.

67
IPCC (2006).
 Glossary 207

Ex-post assessment The process of assessing historical impacts of a policy (i.e. a backward-looking
assessment)

Ex-post baseline scenario A backward-looking baseline scenario that is established during or after
implementation of a policy

Impact assessment The qualitative or quantitative assessment of impacts resulting from a policy, either
ex-ante or ex-post

Impact category A type of sustainable development impact (environmental, social or economic)

affected by a policy

Implemented policies Policies that are currently in effect, as evidenced by one or more of the following:
(1) relevant legislation or regulation is in force, (2) one or more voluntary
agreements have been established and are in force, (3) financial resources have
been allocated, (4) human resources have been mobilized

Independent policies Policies that do not interact with each other, such that the combined effect of
implementing the policies together is equal to the sum of the individual effects of
implementing them separately

Indicator For quantitative impact assessment, a metric that can be estimated to indicate
the impact of a policy on a given impact category. For monitoring performance
over time, a metric that can be monitored over time to enable tracking of changes
towards targeted outcomes.

Indicator value The value of an indicator. For example, 500 is an indicator value for the indicator
“number of jobs created”.

In-jurisdiction impacts Impacts that occur inside the geopolitical boundary over which the implementing
entity has authority, such as a city boundary or national boundary

Intended impacts Impacts that are intentional, based on the original objectives of the policy. In some
contexts, these are referred to as primary impacts.

Interacting policies Policies that produce total effects, when implemented together, that differ from
the sum of the individual effects had they been implemented separately

Intermediate impacts Changes in behaviour, technology, processes or practices that result from a policy,
which lead to sustainable development impacts

Jurisdiction The geographic area within which an entity’s (such as a government’s) authority is
exercised

Life cycle impacts Changes in upstream and downstream activities, such as extraction and production
of energy and materials, or effects in sectors not targeted by the policy, resulting
from the policy

Long-term impacts Impacts that are more distant in time, based on the amount of time between
implementation of a policy and its impacts

Macroeconomic impacts Changes in macroeconomic conditions – such as GDP, income, employment or

structural changes in economic sectors – resulting from a policy

Market impacts Changes in supply and demand, prices, market structure or market share resulting
from a policy
208 Sustainable Development Methodology

Model uncertainty Uncertainty resulting from limitations in the ability of modelling approaches,
equations or algorithms to reflect the real world

Monitoring period The time over which the policy is monitored, which may include pre-policy
monitoring and post-policy monitoring in addition to the policy implementation
period

Negative impacts Impacts that are perceived as unfavourable from the perspectives of decision
makers and stakeholders

Net impact The aggregation of all impacts, both positive and negative, within a given impact
category

Non-policy drivers Conditions other than policies, such as socioeconomic factors and market forces,
that are expected to affect the impact categories included in the assessment
boundary. For example, energy prices and weather are non-policy drivers that
affect demand for heating.

Other policies or actions Policies, actions and projects – other than the policy or action being assessed – that
are expected to affect the impact categories included in the assessment boundary

Out-of-jurisdiction impacts Impacts that occur outside the geopolitical boundary over which the implementing
entity has authority, such as a city boundary or national boundary

Overlapping policies Policies that interact with each other and that, when implemented together,
have a combined effect that is less than the sum of their individual effects when
implemented separately. This includes both policies that have the same or
complementary goals (e.g. national and subnational energy efficiency standards
for appliances) and counteracting or countervailing policies that have different or
opposing goals (e.g. a fuel tax and a fuel subsidy).

Parameter A variable or other type of data needed to calculate the value of an indicator, in
cases where the indicator value cannot be directly measured

Parameter uncertainty Uncertainty regarding whether a parameter value used in the assessment
accurately represents the true value of the parameter

Parameter value The value of a parameter. For example, 5 is a parameter value for the parameter
“tonnes of SO2 emitted per kWh of electricity”.

Peer-reviewed Literature (such as articles, studies or evaluations) that has been subject to
independent evaluation by experts in the same field before publication

Planned policies Policy options that are under discussion, and have a realistic chance of being
adopted and implemented in the future, but have not yet been adopted or
implemented

Policy or action An intervention taken or mandated by a government, institution or other entity,

which may include laws, regulations and standards; taxes, charges, subsidies and
incentives; information instruments; voluntary agreements; implementation of
technologies, processes or practices; and public or private sector financing and
investment

Policy implementation period The time period during which a policy is in effect
 Glossary 209

Policy scenario A scenario that represents the events or conditions most likely to occur in the
presence of a policy (or package of policies) being assessed. The policy scenario is
the same as the baseline scenario except that it includes the policy (or package of
policies) being assessed.

Positive impacts Impacts that are perceived as favourable from the perspectives of decision makers
and stakeholders

Propagated parameter The combined effect of each parameter’s uncertainty on the total result
uncertainty

Proxy data Data from a similar process or activity that are used as a stand-in for the given
process or activity

Qualitative assessment An approach to impact assessment that involves describing the impacts of a policy
on selected impact categories in numerical terms

Qualitative assessment The scope of the qualitative assessment in terms of the range of dimensions,
boundary impact categories and specific impacts that are included in the qualitative
assessment

Quantitative assessment An approach to impact assessment that involves estimating the impacts of a policy
on selected impact categories in quantitative terms

Quantitative assessment The scope of the quantitative assessment in terms of the range of dimensions,
boundary impact categories, specific impacts and indicators that are included in the
quantitative assessment and estimated

Regression analysis A statistical method for estimating the relationships among variables – in
particular, the relationship between a dependent variable and one or more
independent variables

Reinforcing policies Policies that interact with each other and that, when implemented together,
have a combined effect greater than the sum of their individual effects when
implemented separately

Scenario uncertainty Variation in calculated emissions resulting from methodological choices, such as
selection of baseline scenarios

Sensitivity analysis A method to understand differences resulting from methodological choices and
assumptions, and to explore model sensitivities to inputs. The method involves
varying the parameters to understand the sensitivity of the overall results to
changes in these parameters.

Short-term impacts Impacts that are nearer in time, based on the amount of time between
implementation of a policy and its impacts

Specific impact A specific change that results from a policy or action (within a given impact
category)

Stakeholders People, organizations, communities or individuals who are affected by, and/or who
have influence or power over, a policy

Static A descriptor for a parameter that does not change over time
210 Sustainable Development Methodology

Sustainable development Changes in environmental, social or economic conditions that result from a policy,
impacts such as changes in economic activity, employment, public health, air quality and
energy independence

Technology impacts Changes in technology, such as design or deployment of new technologies,

resulting from a policy

Top-down data Macro-level statistics collected at the jurisdiction or sector level, such as energy
use, population, GDP or fuel prices

Trade impacts Changes in imports and exports resulting from a policy

Uncertainty (1) Quantitative definition: Measurement that characterizes the dispersion

of values that could reasonably be attributed to a parameter. (2) Qualitative
definition: A general term that refers to the lack of certainty in data and
methodological choices, such as the application of non-representative factors or
methods, incomplete data or lack of transparency.

Unintended impacts Impacts that are unintentional based on the original objectives of the policy. In
some contexts, these are referred to as secondary impacts.
References

Arksey, Hilary, and Peter T. Knight (1999). Interviewing Analysis: a Manual. Chapter 6. London. Available
for Social Scientists. London: Sage. at https://assets.publishing.service.gov.
uk/government/uploads/system/uploads/
Boonekamp, P. 2006. Actual interaction effects attachment_data/file/7612/1132618.pdf.
between policy measures for energy efficiency:
a qualitative matrix method and quantitative Del Río, Pablo, and others (2013). Interactions
simulation results for households. Energy, vol. 31, between EU GHG and Renewable Energy Policies:
No. 14, pp. 2848–2873. How Can They Be Coordinated? Available at
www.res-policy-beyond2020.eu/pdffinal/
Carbon Brief (2017). Q&A: The social cost of carbon. Interactions%20between%20EU%20GHG%20
Available at www.carbonbrief.org/qa-social-cost- and%20Renewable%20Energy%20Policies%20
carbon. %E2%80%93%20how%20can%20they%20be%20
coordinated%20(beyond2020%20-%20D6-1b).pdf.
Clear Air Task Force (2001). Cradle to Grave: the
Environmental Impacts from Coal. Boston. Available DFID (United Kingdom Department for International
at www.catf.us/wp-content/uploads/2019/10/ Development) (2014). Assessing the Strength of
Cradle_to_Grave-300.pdf. Evidence. London. Available at www.gov.uk/
government/publications/how-to-note-assessing-
ClimateWorks Foundation and World Bank Group the-strength-of-evidence.
(2014). Climate-Smart Development: Adding up
the Benefits of Actions that Help Build Prosperity, Druckman, A., and T. Jackson (2008). Household
End Poverty and Combat Climate Change. San energy consumption in the UK: a highly
Francisco and Washington, D.C. Available at geographically and socio-economically
www-wds.worldbank.org/external/default/ disaggregated model. Energy Policy, vol. 36, No. 8,
WDSContentServer/WDSP/IB/2014/06/20 pp. 3177–3192.
/000456286_20140620100846/Rendered/
PDF/889080WP0v10RE0Smart0Development0Ma. European Commission (2009). Impact Assessment
pdf. Guidelines. Chapter 9. Available at http://
ec.europa.eu/smart-regulation/impact/
Coalition for Evidence-Based Policy (2014). Which commission_guidelines/docs/iag_2009_en.pdf.
Comparison-Group (“Quasi-Experimental”) Study
Designs Are Most Likely to Produce Valid Estimates __________ (no date). Better Regulation Toolbox.
of a Program’s Impact? Washington, D.C. Available Chapter 8: Methods, models, costs, and benefits.
at http://coalition4evidence.org/wp-content/ Available at http://ec.europa.eu/smart-regulation/
uploads/2014/01/Validity-of-comparison-group- guidelines/docs/br_toolbox_en.pdf.
designs-updated-January-2014.pdf.
Federal Office for Spatial Development,
ConnectAmericas (no date). Opportunities in Switzerland (2004). Sustainability Assessment:
the renewable energy value chain. Available Conceptual Framework and Basic Methodology.
at https://connectamericas.com/content/ Bern. Available at https://www.are.admin.
opportunities-renewable-energy-value-chain. ch/dam/are/en/dokumente/nachhaltige_
entwicklung/dokumente/konzept/
Cuesta Claros, Andrea C. (2018). Sustainability impact nachhaltigkeitsbeurteilungrahmenkonzeptfuer
assessment of climate change mitigation policies: eineanwendungautpolit.pdf.
a case study in Mexico. Masters thesis. Helsinki: download.pdf/sustainability_
Aalto University. assessmentconceptualframeworkand
basicmethodology.pdf.
DCLG (United Kingdom Department for Communities
and Local Government) (2009). Multi-Criteria
212 Sustainable Development Methodology

Fullerton, Duncan G., Nigel Bruce and Stephen B. JISEA (Joint Institute for Strategic Energy Analysis)
Gordon (2008). Indoor air pollution from biomass (2014). Multi-Metric Sustainability Analysis. Golden,
fuel smoke is a major health concern in the Colorado. Available at
developing world. Transactions of the Royal Society www.nrel.gov/docs/fy15osti/61719.pdf.
of Tropical Medicine and Hygiene, vol. 102, No. 9,
pp. 843–851. Jungcurt, Stefan (2016). Towards integrated
implementation: tools for understanding linkages
Gold Standard (2014). The Real Value of Robust and developing strategies for policy coherence.
Climate Action: Impact Investment Far Greater than Available at http://sdg.iisd.org/commentary/
Previously Understood. Geneva. Available at www. policy-briefs/towards-integrated-implementation-
goldstandard.org/sites/default/files/documents/ tools-for-understanding-linkages-and-developing-
goldstandard_impactinvestment.pdf. strategies-for-policy-coherence.

HM Treasury, United Kingdom (2011). The Magenta Kentucky Cabinet for Economic Development
Book: Guidance for Evaluation. London. Available (2018). Just the Facts: Economic Impact of 100 Jobs.
at www.gov.uk/government/publications/the- Frankfort. Available at
magenta-book. www.thinkkentucky.com/kyedc/pdfs/100jobs.pdf.

Interagency Working Group on Social Cost of Melamed, Megan L., Julia Schmale and Erika von
Greenhouse Gases, United States Government Schneidemesser (2016). Sustainable policy: key
(2016). Technical Update of the Social Cost of considerations for air quality and climate change.
Carbon for Regulatory Impact Analysis: Under Current Opinion in Environmental Sustainability,
Executive Order 12866. Washington, D.C. Available vol. 23, pp. 85–91. Available at
at https://19january2017snapshot.epa.gov/sites/ https://doi.org/10.1016/j.cosust.2016.12.003.
production/files/2016-12/documents/sc_co2_tsd_
august_2016.pdf. Morra Imas, Linda G., and Ray C. Rist (2009). The
Roads to Results: Designing and Conducting Effective
IPCC (Intergovernmental Panel on Climate Change) Development Evaluations. Washington, D.C.: World
(2006). 2006 IPCC Guidelines for National Bank.
Greenhouse Gas Inventories. Geneva. Available at
www.ipcc-nggip.iges.or.jp/public/2006gl/index. Mulvaney, Dustin (2014). Solar's green dilemma.
html. IEEE Spectrum, vol. 51, pp. 30–33. Available at
https://ieeexplore.ieee.org/document/6882984.
__________ (2007). Policies, instruments, and co-
operative arrangements. In Climate Change 2007: Nerini, Francesco Fuso, and others (2017). Mapping
Mitigation of Climate Change. Contribution of synergies and trade-offs between energy and the
Working Group III to the Fourth Assessment Report Sustainable Development Goals. Nature Energy,
of the Intergovernmental Panel on Climate Change, vol. 3, pp. 10–15. Available at
Bert Metz and others, eds. Cambridge: Cambridge www.nature.com/articles/s41560-017-0036-5.
University Press. Available at www.ipcc.ch/site/
assets/uploads/2018/02/ar4-wg3-chapter13-2.pdf. Nilsson, Måns, Dave Griggs and Martin Visbeck
(2016). Policy: map the interactions between
__________ (2010). Guidance Note for Lead Authors Sustainable Development Goals. Nature, vol. 534,
of the IPCC Fifth Assessment Report on Consistent pp. 320–322. Available at www.nature.com/news/
Treatment of Uncertainties. Available at policy-map-the-interactions-between-sustainable-
https://wg1.ipcc.ch/AR6/documents/AR5_ development-goals-1.20075.
Uncertainty_Guidance_Note.pdf.
NREL (National Renewable Energy Laboratory) (no
ISC (International Science Council) (2017). A Guide to date). JEDI: Jobs & Economic Development Impact
SDG Interactions: from Science to Implementation. models. Available at www.nrel.gov/analysis/jedi.
Paris. Available at https://council.science/
publications/a-guide-to-sdg-interactions-from- Patton, Michael Q. (1987). How to Use Qualitative
science-to-implementation. Methods in Evaluation. Thousand Oaks, CA: Sage
Publications.
 References 213

Sarkar, Soumya (2016). India’s solar dreams, too, Weidema, Bo Pedersen, and Marianne Suhr Wesnaes
are made in China. Quartz India. Available at (1996). Data quality management for life cycle
https://qz.com/india/760079/indias-solar-dreams- inventories: an example of using data quality
too-are-made-in-china. indicators. Journal of Cleaner Production, vol. 4,
No. 3–4, pp. 167–174.
Simmons, Lee (2016). Rare-earth market. Foreign
Policy. Available at https://foreignpolicy. World Bank Group and IHME (Institute for Health
com/2016/07/12/decoder-rare-earth-market-tech- Metrics and Evaluation) (2016). The Cost of
defense-clean-energy-china-trade. Air Pollution: Strengthening the Economic Case
for Action. Washington, D.C. Available at
Solar Foundation (2016). National Solar Jobs http://documents.worldbank.org/curated/
Census 2015. Washington, D.C. Available at en/781521473177013155/The-cost-of-air-
www.thesolarfoundation.org/wp-content/ pollution-strengthening-the-economic-case-for-
uploads/2016/10/TSF-2015-National-Solar-Jobs- action.
Census.pdf.
WRI (World Resources Institute) (2014).
Timmons, Heather (2012). The diesel generator: Greenhouse Gas Protocol Policy and Action
India’s trusty power source. New York Times, Standard. Washington, D.C. Available at
31 July 2012. Available at https://india.blogs. www.ghgprotocol.org/policy-and-action-standard.
nytimes.com/2012/07/31/the-diesel-generator-
indias-trusty-power-source.

UNEP DTU Partnership and IISD (2015). Framework

for Measuring Sustainable Development in NAMAs.
Copenhagen. Available at https://unepdtu.
org/publications/framework-for-measuring-
sustainable-development-in-namas.

USAID (United States Agency for International

Development) (2014). Application of Multi-Criteria
Assessment (MCA) Methods: a Seven Step Process.
Washington, D.C.

U.S. EPA (United States Environmental

Protection Agency (no date, a). Overview
of sophisticated modeling approaches and
tools for state economic analysis. Available at
https://19january2017snapshot.epa.gov/sites/
production/files/2016-03/documents/overview_
modeling_approaches.pdf.

__________ (no date, b). Benefits Mapping and

Analysis Program (BenMAP). Available at
https://19january2017snapshot.epa.gov/benmap/
how-benmap-ce-estimates-health-and-economic-
effects-air-pollution_.html.

USGCRP (United States Global Change Research

Program) (2016). Multi-Scale Economic
Methodologies and Scenarios Workshop. Prepared
by ICF International. Washington, D.C. Available
at www.globalchange.gov/sites/globalchange/
files/reports_files/Multi-Scale%20Economic%20
Medthodologies%20%26%20Scenarios%20
Workshop%20Report_Final_0.pdf.
Contributors

Methodology development leads Dan Forster, Ricardo Energy & Environment

David Rich, World Resources Institute (lead) Denboy Kudejira, independent consultant

Karen Holm Olsen, UNEP DTU Partnership (co-lead) Edward Amankwah, Center for Environmental
Governance

Drafting team Edwin Aalders, DNV GL

Ranping Song, World Resources Institute Eric Zusman, Institute for Global Environmental
Strategies
Alexandra Soezer, United Nations Development
Programme (TWG member) Gajanana Hegde, UNFCCC

Christopher Campbell-Duruflé, Center for Gary Kleiman, independent consultant

International Sustainable Development Law
(TWG member) Grant A. Kirkman, UNFCCC

Denis DR Desgain, UNEP DTU Partnership Hina Lotia, LEAD Pakistan

Fatemeh Bakhtiari, UNEP DTU Partnership Ike Permata Sari, National Standardization Agency of
Indonesia
Gerald Esambe, Green Future Consulting
(TWG member) Jinyoung Park, Korea Transport Institute

Gyanesh K Shukla, independent consultant Kenneth Möllersten, Swedish Energy Agency

(TWG member)
Krista Heiner, EcoAgriculture Partners
Marian Van Pelt, ICF International (TWG member)
Luis Roberto Chacón Fernández, EMA Consulting
Michael Zwicky Hauschild, Technical University of Firm
Denmark, Quantitative Sustainability Assessment
Meinrad Burer, EcoAct
Vikash Talyan, Gold Standard Foundation
(TWG member) Natalie Harms, United Nations Economic and Social
Commission for Asia and the Pacific
Yan Dong, Technical University of Denmark,
Quantitative Sustainability Assessment Olawumi Ayodele Olajide, National Agency for the
Great Green Wall, Nigeria

Technical Working Group Owen Hewlett, Gold Standard Foundation

Alicia González, Aether Sane Zuka, University of Malawi

Ana Rojas, International Union for Conservation of Tanakem Voufo Belmondo, Department of Analysis
Nature and Economic Policies of the Ministry of Economy
and Planning of Cameroon
Arief Wijaya, World Resources Institute – Indonesia
Thomas Damassa, Oxfam
 Contributors 215

Reviewers

Ashwini Hingne, World Resources Institute

Ayushi Trivedi, World Resources Institute

Bodil Jacobsen, Grue + Hornstrup

Chizuru Aoki, Global Environment Facility

Christopher Ede-Calton, World Resources Institute

Claudia Walther, Deutsche Gesellschaft für

Internationale Zusammenarbeit GmbH

David O'Connor, World Resources Institute

Harsha Meenawat, World Resources Institute

Juan Carlos Altamirano, World Resources Institute

Laura Malaguzzi Valeri, World Resources Institute

Pankaj Bhatia, World Resources Institute

Raihan Uddin Ahmed, Infrastructure Development

Company Limited

Richard Pagett, individual

Shenila Parekh, individual

Sun Xia, Institute of International Relations, Shanghai

Academy of Social Sciences

Tanushree Bagh, South Pole Group

Pilot organizations

Aalto University

Initiative for Climate Action and Development

Servicios Ambientales S.A.

UNEP DTU Partnership

University of Cape Town, Energy Research Centre

USAID South Africa Low Emissions Development

(SA-LED) Program and ICF
www.climateactiontransparency.org
[email protected]