unido e va luati on g r o u p

Independent Evaluation
UNIDO-UNEP
Cleaner Production Programme

UNITED NATIONS UNITED NATIONS

INDUSTRIAL DEVELOPMENT ORGANIZATION ENVIRONMENT PROGRAMME
 unido e va luati on g r o u p

Independent Evaluation
UNIDO-UNEP
Cleaner Production Programme

Prepared in cooperation with the

United Nations Environment Programme
Evaluation and Oversight Unit

Supported by:
Austrian Ministry of International and European Affairs
Swiss State Secretariat for Economic Affairs (SECO)

UNITED NATIONS INDUSTRIAL DEVELOPMENT ORGANIZATION

Vienna, 2008
 Distr. GENERAL

OSL/EVA/R.5
15 May 2008

Original: ENGLISH

The views expressed in this Evaluation Report are those of the authors based on their professional assessment
of the evaluation subject. Those views and opinions do not necessarily reflect the views of the Secretariats of
UNIDO and/or UNEP.

The description and classification of countries and territories used, and the arrangements of the material, do not
imply the expression of any opinion whatsoever on the part of the Secretariat concerning the legal status of any
country, territory, city or area, of its authorities, concerning the delineation of its frontiers or boundaries, or
regarding its economic system or degree of development. Designations such as ‘developed’, ‘industrialised’ and
‘developing’ are intended for convenience and do not necessarily express a judgement about the stage reached
by a particular country or area in the development process. Mention of firm names, commercial products and/or
technologies does not imply the endorsement of UNIDO and/or UNEP.

This document has not been formally edited.

Acknowledgement
This programme evaluation of the Cleaner Production Programme of the United Nations
Industrial Development Organisation (UNIDO) and United Nations Environment Programme
(UNEP) was prepared under contract for the UNIDO Evaluation Group/Bureau for
Organisational Strategy and Learning. The evaluation was co-funded by the Governments of
Switzerland and Austria. The Evaluation Team was guided by a Steering Committee,
comprising of:
ƒ Government of Austria (Austrian Development Agency (ADA)): Mr Erwin Kuenzi;
ƒ Government of Switzerland (State Secretariat for Economic Affairs (SECO): Mr Stefan
Denzler;
ƒ UNIDO Evaluation Group: Ms Margareta de Goys (chair);
ƒ UNIDO Energy and Cleaner Production Branch: Mr Heinz Leuenberger;
ƒ UNEP Evaluation and Oversight Group: Mr Segbedzi Norgbey; and
ƒ UNEP Sustainable Consumption and Production Branch: Ms Garrette Clark.

The evaluation team would like to acknowledge the many and diverse contributions made to
this evaluation, by the aforementioned members of the Steering Committee, staff members of
the UNIDO Cleaner Production Unit (in particular Mayra Sanchez, Petra Schwager and Elisa
Tonda), the UNEP Sustainable Consumption and Production Branch (in particular Garrette
Clark, Arab Hoballah and Niclas Sevenningsen) and directors and staff of all National
Cleaner Production Centres, and national consultants that assisted with the country level
evaluations.

Dr Rene Van Berkel (team leader) (1)

Mr Johannes Dobinger (2)
Mr Matthias Meyer (3)
Prof Hans Schnitzer (4)

1
Principal, ECO-INNOVATION, PO Box 523, Inglewood WA 6932, Australia, [email protected]
2
Evaluation Officer, Evaluation Group/Bureau for Organizational Strategy and Learning, UNITED NATIONS
INDUSTRIAL DEVELOPMENT ORGANIZATION, P.O. Box 300, 1400 Vienna, Austria. [email protected]
3
Executive Director, PRAXIMONDO, Rue de l'Arquebuse 10, 1204 Genève, Switzerland,
[email protected]
4
Professor, Institute for Process Engineering, Graz University of Technology, Inffeldgasse 21 B, A-8010 Graz,
Austria, [email protected]

iii
Contents
Glossary viii
Management summary x
Executive summary xiii

1. Introduction 1
1.1 Cleaner Production 1
1.2 Cleaner Production Programme 4
1.3 Independent evaluation 8
1.3.1 Previous evaluations 8
1.3.2 Evaluation methodology 10
1.4 Report overview 13

PART I: FINDINGS

2. Programme review 17
2.1 Overview 17
2.2 Programme design 17
2.2.1 Programme concept 17
2.2.2 Programme strategy and objectives 18
2.2.3 Rationale and logical framework 23
2.2.4 NCPCs and core services 25
2.3 Programme Implementation 26
2.3.1 Programme management 26
2.3.2 National Centres 32
2.3.3 Networking activities 33
2.3.4 Technical assistance 35
2.3.5 Publications and information management 36
2.4 Programme results 36
2.4.1 NCPC 36
2.4.2 Networking 41
2.4.3 Resource materials 43
2.5 Key findings 43
2.5.1 Quality of design 43
2.5.2 Quality of implementation 44

3. Self-evaluation 47
3.1 Introduction 47
3.2 Management information 48
3.3 Service delivery 52
3.3.1 Potential for CP-related service delivery 52
3.3.2 Multilateral Environmental Agreements 58
3.3.3 Resource materials 61
3.4 Self-assessment 65

v
4. Independent country evaluations 67
4.1 Introduction 67
4.2 Country selection 68
4.3 National implementation 71
4.3.1 Preparatory stage 71
4.3.2 Operational stage 74
4.3.3 Programme participation 76
4.4 National results 77
4.4.1 Information dissemination 78
4.4.2 Training 85
4.4.3 Demonstration 86
4.4.4 Policy advice 91
4.4.5 Technology transfer 95
4.5 National assessments 99
4.5.1 Relevance 99
4.5.2 Effectiveness 102
4.5.3 Efficiency 103
4.5.4 Sustainability 105
4.5.5 Capacity development 107
4.5.6 Ownership 110
4.5.7 Overall assessment 112

PART II: ANALYSIS AND ASSESSMENT

5. Portfolio analysis 119

5.1 Introduction 119
5.2 Background 119
5.3 Institutional features 121
5.3.1 Governance 121
5.3.2 Focus 124
5.3.3 Service strategy 126
5.4 Service delivery 129
5.4.1 Information dissemination 129
5.4.2 Training 130
5.4.3 Assessment and demonstration 130
5.4.4 Policy advice 133
5.4.5 Technology transfer 134
5.5 Portfolio and network management 136

6. Programme assessment 143

6.1 Introduction 143
6.2 Uptake of Cleaner Production 143
6.2.1 Relevance 143
6.2.2 Effectiveness 145
6.2.3 Efficiency 148
6.2.4 Sustainability 151

vi
6.3 Capacity development and ownership 154
6.3.1 Capacity development 154
6.3.2 Ownership 156
6.4 Summary assessment 157

PART III: CONCLUSIONS AND RECOMMENDATIONS

7. Conclusions and recommendations 165

7.1 Main conclusions 165
7.2 Detailed conclusions and recommendations 171
7.2.1 Relevance 171
7.2.2 Impact 172
7.2.3 Design strategy 174
7.2.4 Focus 175
7.2.5 Networking 178
7.2.6 Funding model 179
7.2.7 Centre model 181
7.2.8 NCPC services 183
7.2.9 Management and monitoring 185
7.2.10 Administration 186
7.2.11 Governance and ownership 188
7.2.12 Excellence 190
7.3 Final remark 192

Annexes
Annex 1: Bibliography 195
Annex 2: Terms of reference 199

vii
Glossary
ADA Austrian Development Agency
ADB Asian Development Bank
ARSCP Africa Roundtable on Sustainable Consumption and Production
APRSCP Asia Pacific Roundtable on Sustainable Consumption and Production
BAT Best Available Technology
BEP Best Environmental Practice
CL Chemicals Leasing
CP Cleaner Production
CP+ Cleaner Production Plus
CPU Cleaner Production Unit (UNIDO)
CSR Corporate Social Responsibility
DESIRE DEmonstrations in Small Industries for Reducing wastE (India)
DfE Design for Environment
DTIE Division of Industry, Technology and Economics (UNEP)
D4S Design for Sustainability
EE Eco-Efficiency
EECPEMS Energy Efficiency through Cleaner Production and Environmental
Management Systems (GEF)
EERE Energy Efficiency and Renewable Energy
EIA Environmental Impact Assessment
EIP Eco-Industrial Park
EMA Environmental Management Accounting
EMS Environmental Management Systems
EnTA Environmental Technology Assessment
EP3 Environmental Pollution Prevention Project (USEPA and USAID)
ERSCP European Roundtable on Sustainable Consumption and Production
ESTs Environmentally Sound Technology(ies)
FIP Factory Improvement Programme
GC Global Compact
GEF Global Environmental Fund
GERIAP Greenhouse gas Emissions Reduction from Industry in Asia Pacific
HWM Hazardous Waste Management
ILO International Labour Organisation
IRC International Reference Centre
LatinNet Latin-American Cleaner Production Network
LCA Life Cycle Assessment
MDGs Millennium Development Goal(s)
MEAs Multilateral Environmental Agreement(s)
MVA Manufacturing Value Added
NCPCs National Cleaner Production Centre(s)
NCPPs National Cleaner Production Programme(s)
OH&S Occupational Health and Safety
PoI Johannesburg Plan of Implementation

viii
PREMA Profitable Environmental MAnagement
PRISMA Project Industrial Successes with Waste Prevention (The Netherlands)
REAP Responsible Entrepreneur Achievement Programme (UNIDO)
SCP Sustainable Consumption and Production
SDR Sustainable Development Reporting
SECO Swiss State Secretariat for Economic Affairs
SIRM Sustainable Industrial Resource Management
TBL Triple Bottom Line
TEST Transfer of Environmentally Sound Technologies (UNIDO)
UNCED United Nations Conference on Environment and Development
UNEP United Nations Environmental Programme
UNIDO United Nations Industrial Development Organisation
USAID United States Agency for International Development
USEPA United States Environmental Protection Agency
WB World Bank
WBCSD World Business Council for Sustainable Development
WSSD World Summit on Sustainable Development

ix
Management Summary
Since 1994 UNIDO and UNEP cooperate specifically to establish and support National
Cleaner Production Centres/Programmes (NCPCs/NCPPs) in developing countries and
economies in transition. For ease of reference this initiative is throughout this evaluation
referred to as the UNIDO-UNEP Cleaner Production (CP) Programme. In the absence of
a programme document, strictly speaking, however, this is rather a collection of mostly
national and some multi-country projects. Moreover, the roles and responsibilities of both
agencies are in no way equal or comparable in terms of finances, management and
organisational mandate. UNIDO administers the operation of institutionally funded
NCPCs/NCPCs and has the majority of the total resources available for the total
programme. UNEP provides strategic inputs, primarily through separately funded multi-
country projects on emerging topics in Sustainable Consumption and Production (SCP)
and also involves the NCPCs/NCPPs in its series of regional and global strategic
dialogues.

In 2007, the Programme encompassed activities in 37 countries. UNIDO and UNEP view
this CP Programme as a cornerstone of their activities to foster sustainable industrial
development, and agreed to undertake with funding support from the Governments of
Austria and Switzerland, this independent programme evaluation, “to provide conclusive
evidence with regard to the current status, the potential and the needs of the NCPCs and
related initiatives. It will do this by carrying out an independent programme evaluation of
the CP programme, leading to concrete recommendations with regard to the future
strategy of the programme”.

The current status is best summarised as ‘youth’ stage. NCPCs/NCPPs have been
established and are reportedly undertaking CP and CP-related activities. There is a
richness of experience and expertise, and reasonable progress has been made in putting
CP on the agenda, delivering professional training and implementation in particular of
low to medium technology options. There are pockets of excellent results, but also of
poorer quality work, and the Programme has the potential to effectively capture and
disseminate best practices through a strong partnership with the emerging network of CP
support institutions.

The relevance of CP is on the rise, due to worsening industrial pollution, resource

scarcity, globalisation and resulting market pressure and other factors, but the presence
and significance of these trends varies largely between the host countries. Increased
relevance can be expected to lead to higher awareness and demand from public and
private sectors with regard to support for CP services. The remaining gap between the
performance of industry in developing countries and global best practices is considerable
which underlines the relevance and the potential of CP also from a technical perspective.

The biggest challenge for the Programme is to adapt to the changing interests and
demands from governments and private sector. For this, the Programme urgently needs a
consistent Strategy that is impact-focused, delivers and values excellence and takes due

x
account of the specific situation of host countries. The Strategy should drive the
institutionalisation, positioning and profiling of NCPCs/NCPPs into nationally
appropriate niches with customised service and capacity profiles. It should effectively
promote the sharing of leading practices within a competence based network of CP
support institutions, including qualifying NCPCs/NCPPs and other CP service providers
not established through the UNIDO-UNEP CP Programme. The funding, management
and governance models should then also be brought in line with the demands of a
maturing Programme, including more programme and less project-by-project funding and
a truly joint programme management by UNIDO and UNEP. NCPCs will demonstrate
performance against the Programme’s outcomes and impacts to continue their association
with the Programme.

This vision of a strengthened and re-energised Programme has been further expanded in
twelve sets of recommendations:
1. Relevance: the Programme should be continued to assist developing and transition
economies to develop capacity to apply CP practices, technologies, methodologies
and policies in support of their national socio-economic and environmental priorities;
2. Impact: the NCPCs/NCPPs should capitalise on their achievements and target their
service delivery better to increase impact of their services on the uptake of CP
practices, technologies and policies, in particular during the phase of support through
UNIDO-UNEP and donors;
3. Design and Strategy: the Programme should be guided by a succinct programme
document, with a clear strategy, a justification of the intervention logic and the
specific roles and contributions from UNIDO, UNEP and local and international
stakeholders;
4. Focus (Contents): the Programme should re-establish its primary focus on CP and
articulate a dual strategy for its further development to enable specialisation (in
policy and/or technology) and diversification (socially driven and/or environmentally
driven) of NCPCs/NCPPs as they and their national stakeholders see fit in their
respective national contexts;
5. Networking: the Programme should formulate a clear networking strategy with
tangible and realistic outcomes, outputs and activities, which could be realised by
supporting a membership based network that would be open to qualifying institutions,
including NCPCs established by the UNIDO-UNEP CP Programme as well as
eligible other CP service providers;
6. Funding Model: the Programme should adopt a dual funding model at Programme
and national levels: (1) country-based block funding to support NCPCs in their
establishment phase; and (2) programme funding for (i) competitive grants to
multiple eligible NCPCs and possibly qualifying other CP service providers for
project based specialisation and/or diversification; and (ii) networking initiatives;
7. Centre Model: the Programme should articulate institutional objectives and scenarios
for a NCPC so that institutionalisation of the NCPC can be monitored and provisions
be created to accommodate both the public interest and private benefit functions of
the NCPC services over time;
8. NCPC Services: the Programme should support the NCPCs/NCPPs to undertake
periodic assessments of the national status of CP, to define and review their strategic

xi
 niche with service portfolios that are most appropriate and effective in their
respective national contexts;
9. Management and Monitoring: the Programme should adopt a results-based
management model at Programme and national levels and develop a comprehensive
system to monitor performance in capacity building, institutional development and
results and impacts from CP service delivery. It should also monitor that agreed
project structures, governance arrangements and contributions from host countries
and institutions are being achieved.
10. Administration: the Programme management should streamline programme
administration and shift to the extent feasible financial responsibility and
accountability to the NCPCs/NCPPs and/or national stakeholders;
11. Governance and Ownership: the Programme and the NCPCs should adopt
transparent and accountable governance structures at Programme and national levels,
preferably with small boards with participation of private sector, government and
civil society, that assume accountability for the success of the Programme and the
NCPCs; and
12. Excellence: the Programme should establish a culture of experimentation and
continuous improvement in CP service delivery. Sufficient programme funding
should be made available for that purpose.

These main recommendations provide an integrated framework for developing and

managing the UNIDO-UNEP CP Programme for growth and quality of the
NCPCs/NCPPs and related CP initiatives. It is a broad agenda for change that will require
stepwise implementation.

xii
Executive Summary
The United Nations Industrial Development Organisation (UNIDO) and United Nations
Environment Programme (UNEP) cooperate in the promotion of Cleaner Production, with
funding support from various donors, at present in particular the Austrian Ministry of
International and European Affairs and the Swiss State Secretariat for Economic Affairs.
CP is a preventive environmental strategy that can be applied to processes, products and
services to reduce environmental impacts and improve resource productivity.

Since 1994 UNIDO and UNEP cooperate specifically to establish and support National
Cleaner Production Centres/Programmes (NCPCs/NCPPs) in developing countries and
economies in transition. For ease of reference this initiative is throughout this evaluation
report referred to as the UNIDO-UNEP CP Programme. In the absence of a programme
document, strictly speaking, this is rather a collection of mostly national and some multi-
country projects. Moreover, the roles and responsibilities of both agencies are in no way
equal or comparable in terms of finances, management and organisational mandate.
UNIDO administers the operation of institutionally funded NCPCs/NCPCs and has the
majority of the total resources available for the total programme. UNEP provides strategic
inputs, primarily through separately funded multi-country projects on emerging topics in
Sustainable Consumption and Production (SCP) and also involves the NCPCs/NCPPs in
its series of regional and global strategic dialogues.

In 2007, the Programme encompassed activities in 37 countries. UNIDO and UNEP view
this CP Programme as a cornerstone of their activities to foster sustainable industrial
development, and agreed to undertake with funding support from the Governments of
Austria and Switzerland, this independent programme evaluation.

Scope and Methodology (Chapter 1)

This programme evaluation was initiated to document and asses the activities and results
of the NCPCs/NCPPs established by UNIDO in cooperation with UNEP, taking the
historic programme documentation as a reference point. It was also aimed to provide
suggestions and recommendations for strengthening the global network of
NCPCs/NCPPs, for improving service delivery in the host countries and for further
catalysing sustainable industrial development in developing countries and economies in
transition.

The evaluation considered six evaluation criteria, including four primary criteria that
relate to the uptake of CP (respectively: relevance, effectiveness, efficiency and
sustainability) and two secondary criteria that assess two important overall quality
dimensions for development assistance initiatives (respectively: capacity development
and ownership).

The evaluation is based on three information sources, respectively: review of programme

and its management; self-evaluations of the 38 current NCPCs/NCPPs, and independent

xiii
country evaluations for 18 NCPCs5. The findings were considered in an integrated
manner to: analyse the diversity in programme implementation at the national levels
(‘portfolio analysis’); assess the Programme against the evaluation criteria; and provide
overall conclusions and recommendations.

The evaluation was executed between April and December 2007, by an international
expert team, assisted by national consultants in the visited countries, operating under the
guidance of a Steering Committee of UNIDO, UNEP and donor representatives. Interim
results including draft conclusions and recommendations were presented for review to the
9th Annual Meeting of NCPC Directors, held in Semmering (Austria) on 24-26 September
2007. A comprehensive draft was released in January 2008. The report was then finalised
in April 2008 taking into due consideration the comments and suggestions from UNIDO,
UNEP and donor representatives.

Programme Review (Chapter 2)

The explicit and implicit objectives of the UNIDO-UNEP CP Programme were reviewed,
and activities of UNIDO and UNEP to achieve those objectives were analysed.

It was found that the CP concept is well reflected in the Programme and that the original
Programme was a coherent approach to building CP into an international cooperation
initiative. The consistency and clarity of the Programme have diminished over time as a
result of the repeated attempts to re-design and re-shape the Programme that were only
partially incorporated into national project plans and lacked a clear vision and logical
framework for the Programme as a whole. The NCPC model is largely successful, given
its replication within and outside the Programme, and continued demand for the set up of
new NCPCs. Cooperation between UNIDO and UNEP as well as networking among
NCPCs/NCPPs have not yet been designed into the Programme. There is also no strategy
to deal with NCPCs that are no longer funded through the UNIDO-UNEP CP
Programme.

The Programme started with a programmatic approach which included a generic

cooperation agreement between UNIDO and UNEP, a programme document for
establishing NCPCs in five countries and a competence based application process for
establishing these first NCPCs. Over time this weakened considerably in favour of
management of individual CP projects (predominantly to set up or support one, or several
co-located, NCPC(s)) with little steering and monitoring at programme level. The
approach has been successful in establishing NCPCs/NCPPs. It limited however the
potential to learn from past and parallel experience within the Programme to improve
quality and effectiveness of CP interventions (including projects not exclusively related to
NCPCs) and build and exercise professional and thematic leadership in CP.

The Programme has used a select group of CP service providers to act as International
Reference Centres (IRCs) to the NCPCs/NCPPs. This has been beneficial for fostering

5
Country reports will be made available by UNIDO Evaluation Group upon request.

xiv
coherence in programme implementation among recipient countries, and the use of more
experienced NCPCs as IRCs for newly established NCPCs/NCPPs is being applauded.
With the maturing of the Programme, more attention is needed to expose NCPCs/NCPPs
to different methods and practices for CP service delivery, and thereby enable
NCPCs/NCPCs to develop methods and practices that are most suited to the local
circumstances in their home countries.

The limited internal (within UNIDO) and external (inter-agency) cooperation presents a
barrier for wider impact at programme level. These shortcomings were in part outside the
control of the CP Programme due to systemic constraints within the UNIDO management
and administrative systems, leading to a project-by-project approach and a general lack of
programme-based funding.

Self Evaluation (Chapter 3)

The self evaluation was undertaken to obtain comparable baseline information on the
operation, management and activities of all NCPCs/NCPPs directly from the Directors in
charge of running these on a daily basis. It was executed by means of two surveys, one on
operational, institutional and managerial aspects of the NCPC (completed by 36
NCPCs/NCPPs, i.e. response rate 95%) and one on emerging topics and tools and
available resource materials within the UNIDO-UNEP CP Programme (completed by 23
NCPCs/NCPPs, i.e. response rate of 61%).

The majority of NCPCs/NCPPs operates with limited independence, either as subsidiary

of their host organisation (formally or informally as an administratively and financially
isolated activity area) or otherwise semi-autonomously, with only some 30% being fully
independent. They therefore typically assume the legal status of their host institutions,
which in about half of the countries is a public sector entity and in some 10% of the
countries a private sector institution. About 30% of the NCPCs/NCPPs describe their
legal status as unresolved.

Just over 80% of the responding NCPCs/NCPPs reported to have received some
institutional funding through the UNIDO-UNEP CP Programme, most often for 3 to 4
years, even though several centres have been funded for much longer. The accumulated
funding amounts reported by the Directors vary widely (70-fold), with an average of
some USD 863,000 per country. Reported annual budgets for the NCPC/NCPP vary
between USD 50,000 and USD 3,600,000, with an average (excluding the lowest and
highest outlying values) of USD 463,000. The average percentage contribution to the
operating budget of all NCPCs/NCPPs is 28% from UNIDO-UNEP CP Programme, 26%
from private sector, 23% from other donor programmes, and 18% from national
governments. The average staff strength (upon exclusion of the outlying lower and higher
values) is 11.3 full time equivalent, comprising 1.9 in management, 6.9 at professional
level and 2.5 at administrative and support levels.

The activity information confirmed that three of the Programme’s key CP services are
provided by at least 80% of the NCPCs/NCPPs, respectively: information dissemination,

xv
training and CP assessments (and/or in plant demonstrations). The two other service areas
(policy advice and technology transfer) are delivered by about half of the NCPCs/NCPPs.
About one third of the NCPCs/NCPPs delivers other services, most commonly related to
Occupational Health and Safety (OH&S), Environmental Impact Assessment (EIA), Life
Cycle Assessment (LCA) and/or Design for Environment/Sustainability (DfE/D4S).
There is general agreement for the potential for service delivery in some CP-related
fields, in particular OH&S, Energy Efficiency and Renewable Energy (EERE),
Hazardous Waste Management (HWM), Environmental Management Systems (EMS)
and Environmental Technology Assessment (EnTA). About three quarters of the
NCPCs/NCPPs claims to have expertise in these areas, except for OH&S. In regard to key
Multilateral Environmental Agreements (MEAs), with the exception of the Marrakech
process on Sustainable Consumption and Production (SCP), reported expertise and
involvement is relatively low and patchy among the NCPCs/NCPPs.

The Directors also self-assessed their NCPC/NCPP against five of the evaluation criteria,
respectively: relevance, effectiveness, efficiency, sustainability and ownership (see Table
S1). The responses indicated a high level of confidence from the Directors that their
NCPC/NCPP performs well. The self-assessment is most optimistic about relevance and
effectiveness, rated ‘high’ by respectively 67% and 61% of the respondents. The self-
assessment is also good for efficiency, rated ‘high’ and ‘medium’ by respectively 50%
and 25% of the respondents. It would appear that there is some more doubt about
performance on sustainability and ownership, with the ‘high’ scores for self assessments
falling to 39% and 28% of respondents and ‘medium’ ratings increasing to respectively
36% and 39%.

Table S1: Self assessment against evaluation criteria (36 responses)

Evaluation Self Assessment Rating
Criteria High Medium Low Unknown or Total
No Response
1. Relevance 24 67% 7 19% 1 3% 4 11% 36 100%
2. Effectiveness 22 61% 8 22% 0 0% 6 17% 36 100%
3. Efficiency 18 50% 9 25% 2 6% 7 19% 36 100%
4. Sustainability 14 39% 13 36% 3 8% 6 17% 36 100%
5. Ownership 10 28% 14 39% 8 22% 4 11% 36 100%

Independent Country Evaluations (Chapter 4)

Independent evaluation missions were undertaken to obtain first hand information from
the Director and staff of the NCPC, members of its board, national government agencies,
industry associations and clients of NCPC services.

The countries were selected with a view to achieve maximum diversity among countries
to be evaluated in detail, in regard to location, donor for and maturity of the NCPC and
size/structure of the national economy. The final sample was endorsed by the Steering
Committee and included 18 countries, respectively China, Columbia, Costa Rica, Croatia,
Egypt, El Salvador, Guatemala, India, Kenya, Mexico, Morocco, Mozambique,

xvi
Nicaragua, Peru, South Africa, Sri Lanka, Uzbekistan and Vietnam. Even though the
results from these 18 countries are considered illustrative for the status of the Programme,
they are not representative for the entire Programme due to non-randomised country
selection and potential differences in the interpretation of data and judgements by
individual evaluators.

For the visited countries, the activities of Programme management and NCPCs were
reviewed, including the establishment and operational stages and the participation of the
NCPCs in the global Programme. It was found that the establishment stages have been
dominated by the fund raising, leading to minimalist approaches to project justification
and feasibility analysis. It was also found that in the operational stages there were
shortcomings in regard to transparency and accountability of governance (in particular to
national stakeholders) and professionalisation of service delivery of the NCPCs/NCPPs
across all their service areas. Moreover, there is no provision for ongoing interaction with
NCPCs that are no longer institutionally funded through the Programme. The
NCPCs/NCPPs and their national stakeholders remain loyal to the Programme, but there
is a strongly felt need to streamline Programme administration and to increase the
availability and intensity of networking opportunities within the Programme.

The national results in regard to the five core service areas were also analysed. Schemes
were established to classify and compare results between countries. In over 75% of the
visited countries outputs were substantive for three service areas (information
dissemination technology transfer and CP assessments). This was markedly lower for
policy advice (some 60% of countries) and training (some 50% of countries). Generally
achievements in terms of outcomes are less substantive and data availability in regard to
outcomes and particularly impacts is very limited. In spite of that, there is typically
reasonable ground to confirm some positive outcomes, which in turn is a weak leading
indicator for impact. There is however not always a causal link between level of output
and level of outcomes, as outcomes have in some countries been achieved through non-
NCPC activities.

The 18 visited NCPCs were also assessed on the six evaluation criteria by the
independent evaluators. Figure S1 shows the frequency distributions of all countries. The
distributions are quite similar for the

xvii
Figure S1: Summary of results of national level evaluation on programme level evaluation criteria
Summary of National Assessments

100% 0 0 0 0
2 2
3 3
90% 4
6
80%
5
6
70%
6
7
countries (n=18)

60% 6

5
50%

40% 6 8
5 3
30% 5
5
20%
4

10% 4 4
3 3
2
1
0% 0
Relevance Effectiveness Efficiency Sustainability Capacity Ownership
Development
Evaluation Criteria

excellent good satisfactory poor absent

four primary evaluation criteria. The highest score is achieved for sustainability, closely
followed by effectiveness, efficiency and relevance. For each of these criteria 39% to
56% of the NCPCs achieved a score in either of the two highest assessment categories
(‘excellent’ or ‘good’). The evaluation on the secondary criteria is markedly weaker, as
just 16% and 28% of the visited countries, respectively on capacity development and
ownership, attained either of two highest assessment categories.

Portfolio Analysis (Chapter 5)

The portfolio analysis reviewed similarities and differences in the establishment and
operation of NCPCs/NCPPs to gain a better understanding of the current richness and
diversity in the CP Programme and identify possible avenues to bolster these as the
Programme develops further. The current diversities at the national level are a result of
internal factors (those controlled or at least to a considerable degree controllable by the
Programme, including centre-, project- and programme-factors) and external factors
(those that are not under the control of the Programme but that the Programme can adapt
to, including state of environment and the economy and status of knowledge). The
portfolio analysis was complemented with suggestions for further development of
concept, methods, tools and institutional arrangements for the Programme which illustrate
how the findings of the analysis shed new light on the design, strategy, management and
administration of the Programme.

At the Centre level the portfolio analysis found that even though some kind of governance
structures exist for most NCPCs/NCPPs, considerable scope exists for better governance
to improve transparency and accountability of decision making, in particular to national
stakeholders, equally from the private and public sectors.

xviii
The UNIDO-UNEP CP Programme has over time expanded its scope to include new
topics and tools, but a major weakness remains that these have essentially been ‘added
on’ instead of ‘integrated into’ the key service areas and core CP concepts. It is being
suggested that a distinction be made in ‘specialisation’ (improving the rigour and depth
of service delivery related to CP implementation, for policy and/or technology) and
‘diversification’ (introducing services pertaining to topics related to CP, for CSR and/or
SCP). NCPCs will also have to adapt, and some have started doing so, to the fact that
other national institutions have considerable CP capacities, so that the traditional CP
services may no longer be appropriate and/or sufficient. It is being suggested to
differentiate in service tiers, respectively: audit and training services (Tier 1); technology
and policy development services (Tier 2); and networking services (Tier 3). Each NCPC
can develop its own niche, in regard to the balance of its capabilities among core,
specialised and diversified CP topics, as well as balance between Tier 1, Tier 2 and Tier 3
services.

The portfolio analysis found relatively minor differences among NCPCs/NCPPs in regard
to information dissemination and training, even though there remains a need for
developing strategies to maximise the impact of these services, including through the
adoption of best professional practices. The approaches to service delivery in the other
three core services are quite different between the NCPCs/NCPPs. In regard to CP
assessments standardisation and professionalism within each NCPC/NCPP deserve
improvement whilst there is also potential to improve service delivery through concepts
and methods that are customised to national circumstances. For policy advice, the degree
of pro-activeness of NCPCs/NCPPs differs quite substantially. Overall there is an
opportunity to expand the scope of policy advice beyond the traditional environmental
policy domain, to cover economic and technology domains. Only some NCPCs have
substantial experience in developing and delivering technology transfer services. It is
suggested that current leading insights in EST transfer are used to develop a balanced and
integrated set of programme activities on EST transfer within the UNIDO-UNEP CP
Programme.

As NCPCs/NCPPs find their strategic niches in their respective national contexts the
UNIDO-UNEP CP Programme will change qualitatively. This can be fostered by
supportive changes in funding and network management. The funding basis could be split
between block funding (for specific countries for establishment of a NCPC) and
competitive grant funding (for eligible NCPC to develop and deliver specific activities or
services), with the balance between the two shifting towards competitive grants over the
life-time of each NCPC. The network could be managed as a membership-based
association of CP service providers, with different membership categories having to meet
different membership criteria and having different rights and obligations, including the
ability to benefit from services and funding through the UNIDO-UNEP CP Programme.

xix
Programme Assessment (Chapter 6)

The Programme as a whole has been assessed by the international evaluation team on all
six evaluation criteria, using itemised scorecards based on the various programme
documents. The assessments itself are based on the three main sources of information
(programme review, self evaluation and independent country evaluations). The overall
result is presented in Figure S2. The variation in the averaged programme level
assessment scores for the six evaluation criteria is relatively limited. Sustainability and
relevance have the highest scores (respectively 3.0 and 2.9), followed by effectiveness,
efficiency and capacity building (respectively 2.5, 2.5 and 2.4), and then followed by
ownership (score of 1.3). Figure S2 shows that the programme assessments are in the
range of being satisfactory. Given the high ambitions, complexity and scope of the
UNIDO-UNEP CP Programme this should be regarded as a good assessment result.

Figure S2: Averaged programme-level assessment for all evaluation criteria

Programme Scores: 1 = unsatisfactory, 2 = low, 3 = satisfactory, 4 = good and 5 = excellent

Programme Assessment

4
averaged programme scores

0
relevance effectiveness efficiency sustainability capacity ownership
development

This programme assessment is underpinned by the following key findings.

1. CP is of continued and rising relevance.

CP is generally considered relevant by government, private sector and other stakeholders
in host countries for the UNIDO-UNEP CP Programme. Several current global trends
cause the relevance of CP to rise, but the presence and significance of these trends varies
greatly between the host countries.

xx
2. The UNIDO-UNEP CP programme has produced valuable outputs and outcomes
in all 18 countries visited for an independent evaluation.
Its principal achievement has been in putting CP on the agenda of government and
business, building capacity for CP, development of information materials, implementation
of good housekeeping and low/intermediate technology options in selected companies
and policy change in some countries.

3. The potential of the CP Programme has not been fully exploited.

The country visits demonstrated that each NCPC is unique in its institutional setting,
activities and achievements, with considerable differences from the ‘idealised’ NCPC as
being portrayed by the UNIDO-UNEP CP Programme and advocated by its management.
The Programme has not yet demonstrated flexibility to sufficiently adapt its support to the
specific needs and activities of the different countries and to enable different types of
NCPCs to fulfil niche roles that are most appropriate and effective in their specific
national contexts. The absence of programme-based funding has contributed to a scattered
approach to networking and learning, with limited opportunities for capturing and
advancing best practices and for strengthening and managing the network.

4. Design and strategy of the CP Programme have major shortcomings.

There is no over-arching programme document. The overall objectives of the UNIDO-
UNEP CP Programme are therefore not always explicit causing stakeholders’
expectations of the Programme to vary. A logical means-end relationship between the
overall objectives, impacts, outcomes and outputs, and activities of the Programme has
not been established, which has led to a rather standardised approach for the introduction
of CP on a project-by-project basis and to a lack of demand-based models for national
implementation of the Programme that customise to the unique national institutional set
up and capability portfolios of each of the Centres.

5. Weak monitoring and reporting limits adaptive and effective programme

management.
Monitoring of outcomes and impacts is generally insufficient to allow reporting of
Programme achievements against Programme objectives. This hinders adaptive
management and continuous improvements in service delivery, at national and
programme levels.

6. The ‘win-win’ premise of CP is true only under specific circumstances.

The ‘win-win’ premise on which the UNIDO-UNEP CP Programme is largely based is
not universally achievable in the host countries. The continued reference to this premise
has created expectations among national stakeholders that cannot be met and in turn
weakened their buy-in into the Programme.

7. The UNIDO-UNEP CP Programme was not very successful in EST Transfer

Some CP technology investments have been facilitated through the Programme, often by
utilising available green credit lines and/or deployment of local engineering design and
fabrication capacities. Overall however the Programme has made little headway in

xxi
transferring ESTs, neither through the regular activities of the NCPCs nor through
specific CP technology transfer initiatives.

8. Creation of NCPCs/NCPCs is an appropriate way for capacity building in CP but

attention for their institutionalisation has been limited.
The UNIDO-UNEP CP Programme has defined NCPCs by their portfolio of standardised
CP services. The institutional dimension of the NCPCs (e.g. the NCPC’s role vis-à-vis
other types of institutions, the NCPC’s role in the national innovation system) has
therefore not been sufficiently considered in many cases.

9. The potential for cooperation with other initiatives has not been exploited.
The evaluation found only limited evidence of ongoing collaborations within the UN
agencies, with other UN Agencies, with donors other than the ‘current’ UNIDO CP
Programme donors, and with other initiatives in the field of industry, environment and
sustainability. Given the multitude of such initiatives, there is an unexploited potential to
leverage expertise and resources at the programme and national levels.

10. The valuable contribution of the programme to national capacity building is not
sufficiently communicated.
UNIDO, UNEP and Donors have a tendency to present NCPCs as ‘their’ institutions,
despite of their national ownership and governance structures, substantially different
activity portfolios and funding models. This ignores the fact that many NCPCs do no
longer have a close relationship with the CP Programme and does not reflect the role of
the Programme in building up and supporting national capacities and ownership.

11. There is a trade-off between financial independence and sustained impact.

The evaluation showed that the sustainability of the Programme’s achievements in
building CP capacity, implementing CP in companies and CP-promoting policies is
generally high. It is however noted that the priority assigned to financial sustainability (or
rather independence) of the NCPC as a national institution (largely through income from
services) can become counterproductive to achieving sustained effects and impacts as
measured by the Programme’s objectives.

Conclusions and Recommendations (Chapter 7)

The evaluation team found that relevance and sustainability of the UNIDO-UNEP CP
Programme are good, with scope for improvement particularly for effectiveness and
efficiency, which could result in better targeted, customised and streamlined interventions
at the national level, which in turn could further bolster relevance and sustainability, as
well capacity development and ownership. The conclusions and recommendations are
organised in twelve clusters, respectively: relevance; impact; design and strategy; focus;
networking; funding model; centre model; NCPC services; management and monitoring;
administration; governance and ownership; and excellence. The main recommendations
of these clusters provide an integrated framework for developing and managing the
UNIDO-UNEP CP Programme for growth, impact and maturity of the NCPCs/NCPPs.
The twelve clusters with their main conclusion and overarching recommendation are

xxii
provided in Table S2. Detailed supportive conclusions and recommendations are provided
for each cluster (see Section 7.2).

Table S2: Overview of main conclusions and overarching recommendations

Cluster Main Conclusion Overarching Recommendation
1. Relevance CP is relevant and its relevance The CP Programme should be continued
is on the rise due to worsening to assist developing and transition
industrial pollution, resource economies to develop capacity to apply
scarcity, entering into force of CP practices, technologies,
MEAs, trade liberalisation and methodologies and policies in support of
globalisation, buyer pressure and their national socio-economic and
greater government and environmental priorities.
community awareness.
2. Impact The Programme was successful The NCPCs/NCPPs should capitalise on
in establishing CP initiatives in their achievements and target their service
each host country and all were delivery better to increase impact of their
reported to be active. For the services on the uptake of CP practices,
visited countries it could be technologies and policies, in particular
confirmed that the NCPC had during the phase of support through
produced valuable outputs and UNIDO-UNEP and donors.
outcomes in particular with
regard to awareness raising,
training, implementation of low
and intermediate technology CP
options and, in some countries,
policy change.
3. Design & There is no programme The Programme should be guided by a
Strategy document covering the overall succinct programme document, with a
objectives, the strategy and clear strategy, a justification of the
intervention logic and the intervention logic and the specific roles
different expected contributions and contributions from UNIDO, UNEP
from UNIDO, UNEP and local and local and international stakeholders.
stakeholders. Existing strategy
documents are not useful for
Programme management.
4. Focus The expansion of the scope of The Programme should re-establish its
(Contents) the CP concept that has primary focus on CP and articulate a dual
gradually occurred in the strategy for its further development to
Programme over time catalysed enable specialisation (in policy and/or
by interests of the donors and the technology) and diversification (socially
UN agencies, is not widely driven and/or environmentally driven) of
understood by all programme NCPCs/NCPPs as they and their national
stakeholders and lacks stakeholders see fit in their respective
widespread endorsement by the national contexts.
NCPCs/NCPPs and their
national stakeholders.

xxiii
Cluster Main Conclusion Overarching Recommendation
5. Networking The Programme has not The Programme should formulate a clear
formulated a distinct strategy networking strategy with tangible and
with tangible objectives, realistic outcomes, outputs and activities,
outcomes and outputs for which could be realised by supporting a
networking among NCPCs and membership based network that would be
the resource needs for its open to qualifying institutions, including
facilitation and technical support NCPCs established by the UNIDO-UNEP
through the UNIDO-UNEP CP Programme as well as eligible other
Programme management have CP service providers
not been identified.
6. Funding The predominant model for The Programme should adopt a dual
Model funding of the Programme as a funding model at Programme and national
collection of country projects levels: (1) country-based block funding to
has hindered effective support NCPCs in their establishment
networking and constrained the phase; and (2) programme funding for (i)
Programme in developing and competitive grants to multiple eligible
delivering specialist services on NCPCs and possibly qualifying other CP
a multi-country basis. service providers for project based
specialisation and/or diversification; and
(ii) networking initiatives.
7. Centre The capacity building model The Programme should articulate
Model through NCPCs/NCPCs is institutional objectives and scenarios for a
relevant, even though the NCPC so that institutionalisation of the
Programme defines NCPCs by NCPC can be monitored and provisions
their service categories without be created to accommodate both the
providing clear institutional public interest and private benefit
perspective(s) for the NCPC, functions of the NCPC services over time.
both during and beyond their
phase of institutional funding
through the UNIDO-UNEP CP
Programme.
8. NCPC The Programme has outlived its The Programme should support the
Services initial design of services which NCPCs/NCPPs to undertake periodic
was based on a standard package assessments of the national status of CP,
of NCPC services to be to define and review their strategic niche
delivered through one single with service portfolios that are most
national centre, as countries that appropriate and effective in their
have built CP capacity in respective national contexts.
different institutions require
more tailor made NCPC
services.

xxiv
Cluster Main Conclusion Overarching Recommendation
9. Manageme Reporting on Programme The Programme should adopt a results--
nt & achievements is generally based management model at Programme
Monitoring insufficient to assess outcomes and national levels and develop a
and impacts against Programme comprehensive system to monitor
objectives which prevents performance in capacity building,
adaptive management and institutional development and results and
continuous improvement of the impacts from CP service delivery. It
Programme’s performance. should also monitor that agreed project
structures, governance arrangements and
contributions from host countries and
institutions are being achieved.
10. Administrat The UNIDO CP Unit and The Programme management should
ion NCPCs/NCPPs have ultimately streamline programme administration and
been able to meet administrative shift to the extent feasible financial
requirements, including financial responsibility and accountability to the
administration and contracts’ NCPCs/NCPPs and/or national
management and disbursement stakeholders.
of funds, but repeatedly not in a
timely manner.
11. Governance The Programme has not The Programme and the NCPCs should
& established a transparent and adopt transparent and accountable
Ownership accountable governance governance structures at Programme and
structure for gathering feed back national levels, preferably with small
from stakeholders, beneficiaries boards with participation of private
and NCPCs into its strategic sector, government and civil society, that
planning and ensuring adequate assume accountability for the success of
oversight over implementation the Programme and the NCPCs.
of the Programme. The
governance of NCPCs is of
varying effectiveness,
accountability and transparency.
12. Excellence Despite its ambition for The Programme should establish a culture
excellence, thematic leadership of experimentation and continuous
in the Programme management improvement in CP service delivery.
is weak, as well as its incentives Sufficient programme funding should be
and opportunities for realising made available for that purpose.
continuous improvements in
development, adaptation and
replication of CP services and
initiatives.

The output of this evaluation study is a sound evidence basis on the status, potential and
needs of the NCPCs/NCPPs as well as practical recommendations and suggestions for
improving the Programme. It is hoped that the planned outcome will now also be forth-
coming, namely: “UNIDO management, UNEP management, donors and other
stakeholders will use the conclusions and recommendations of the evaluation to elaborate
an evidence-based, comprehensive strategy for future assistance to and cooperation with
Cleaner Production Centres and Programmes and related initiatives and institutions”. It

xxv
is understood that the scope of recommendations is broad and that implementation of
recommendations should therefore be undertaken step-by-step.

xxvi
1
Introduction
_______________________________________

1.1 Cleaner Production

The United Nations Environment Programme (UNEP) coined the term Cleaner
Production (CP) at its first International Expert Group Meeting on Preventive
Environmental Strategies in Canterbury in the United Kingdom in 1990. The meeting
coincided with demonstration, research and policy projects on preventive environmental
management strategies (e.g. [1-4]), the emergence of a network of national pollution
prevention programmes (www.p2.org) and the launch of federal and state Pollution
Prevention and Toxic Use Reduction legislation in the USA (e.g. [3, 5, 6]) and the
completion of the first set of waste prevention demonstration projects in Europe (e.g. [7,
8]).

The scope of CP has been subject of much debate in particular in the early 1990s, with a
consolidation by about 1994 into a consensus definition that has since been widely used
within the United Nations System (including United Nations Industrial Development
Organisation (UNIDO)), intergovernmental banks (e.g. World Bank (WB) and Asian
Development Bank (ADB)) and national governments in different parts of the world. The
definition states:

“Cleaner Production is the continuous application of an integrated

environmental strategy to processes, products and services to increase efficiency
and reduce risks to humans and the environment” [9].

A number of related terms exists that for practical purposes can be considered as
essentially equivalents for CP. Table 1.1 provides some examples. The overlap with Eco-
Efficiency (EE) is greatest. Championed by the World Business Council for Sustainable
Development (WBCSD), Eco-Efficiency is best characterised as ‘doing more with less’,
that is using materials, energy and other natural resources more efficiently for the delivery
of more valuable goods and services. In a similar vein, CP can then be characterised as
‘turning waste to profit’, that is eliminating waste and pollutants at source to reduce
environmental impacts [10]. CP can notionally be measured with a ratio of units of
pollution or resource use per unit of production (or Manufacturing Value Added, MVA).
EE can be measured by the inverse ratio of units of production per unit of pollution and/or
resource use [11]

1
Table 1.1: Examples of CP-related terminology
Term Definition
Eco-Efficiency The delivery of competitively priced goods and services that satisfy human
needs and bring quality of life, while progressively reducing ecological impacts
and resource intensity throughout the life-cycle, to a level at least in line with the
earth’s estimated carrying capacity [12]
Green A broad strategy for enhancing productivity and environmental performance and
Productivity leading to positive change in socio-economic development [13]
Pollution Reducing or eliminating waste at the source by modifying production processes,
Prevention promoting the use of non-toxic or less-toxic substances, implementing
conservation techniques, and re-using materials rather than putting them into the
waste stream [6]
Waste Application of a systematic approach to reducing the generation of waste at
Minimisation source…. It is about optimising all areas of the business to be more resource
efficient and thus prevent, or at least minimise, the production of waste [14]

CP combines technological and organisational dimensions. There is a tendency to view

the technological part or the cleaner process technologies (i.e. production technologies
that are inherently less resource intensive or less wasteful) as a subset of Environmentally
Sound Technologies (ESTs). EST refers to a set of technologies that is applied to
deliver environmental benefits. Chapter 34 of Agenda 21 provides a basic
definition [15]:

“ESTs protect the environment, are less polluting, use all resources in a
more sustainable manner, recycle more of their wastes and products, and
handle residual wastes in a more acceptable manner than the technologies
for which they were substitutes (34.1)”

ESTs include other subsets for example end-of-pipe (or clean-up) technologies,
renewable energy technologies, etc. Even though these are supportive of achieving CP
outcomes, they are not commonly understood as being part of core CP.

CP is strongly embedded in international environmental and sustainable development

policies and strategies.

ƒ Chapter 30 of Agenda 21 [15] calls upon national governments, industry and

international organisations to collaborate on the dissemination and implementation of
CP technologies and practices
(http://www.un.org/esa/sustdev/documents/agenda21/english/agenda21chapter30.htm).

ƒ CP is well aligned with the Millennium Development Goals (MDG), in particular

MDG 7 (ensuring environmental sustainability)
(http://www.un.org/millenniumgoals/). Implementation of CP can, on a case by case
basis, also contribute to other MDGs, in particular MDG 1 (reducing poverty, for
example when CP enhances productivity leading to more jobs, or reduction of
environmental health burden on the poor), MDG 3 (promote gender equality and
empower women, for example when CP empowers working women and improves
their work environment) and MDG 8 (developing a global partnership for
development, for example where government, private sector and community
collaborate on CP to foster development).

2
ƒ CP is also supportive of the Global Compact, in particular for the environmental
principles, respectively: business should support a precautionary approach to
environmental challenges (principle 7); business should undertake initiatives to
promote greater environmental responsibility (principle 8) and business should
encourage the development and diffusion of environmentally friendly technologies
(principle 9)
(http://www.unglobalcompact.org/AboutTheGC/TheTenPrinciples/environment.html)
.

ƒ The Johannesburg Plan of Implementation (PoI) [16] provides the most recent
endorsement for CP. Chapter 3 positions CP in the framework of “changing
unsustainable patterns of consumption and production”. Paragraph 15 calls to
“increase in investment in cleaner production and eco-efficiency in all countries,
through inter alia, incentives and support schemes and policies directed at
establishing appropriate regulatory, financial and legal frameworks. This would
include actions at all levels to establish and support cleaner production programmes
and centres and more efficient production methods by providing, inter alia, incentives
and capacity building to assist enterprises, especially small and medium sized
enterprises and particularly in developing countries, in improving productivity and
sustainable development” (3.15)
(http://www.un.org/esa/sustdev/documents/WSSD_POI_PD/English/POIChapter3.ht
m).

The Johannesburg World Summit on Sustainable Development (WSSD) has markedly

expanded the mandate of UNEP from CP into Sustainable Consumption and Production
(SCP). Within the United Nations System, UNEP is the custodian of the global effort to
“develop 10 year frameworks of programmes in support of national initiatives to
accelerate the shift towards sustainable consumption and production to promote social
and economic development within the carrying capacity of ecosystems by addressing and,
where appropriate, delinking economic growth and environmental degradation through
improving efficiency and sustainability in the use of resources and production processes,
and reducing resource degradation, pollution and waste” (paragraph 14, chapter 3)
(http://www.un.org/esa/sustdev/documents/WSSD_POI_PD/English/POIChapter3.htm).
This effort on 10 year framework programmes is commonly referred to as the ‘Marrakech
process’.

Considerable effort has been devoted to develop a further understanding of SCP in local
and regional context. A recent summary states that [17]: “Changing consumption and
production patterns towards more sustainable ones means improving:
ƒ The technologies (or in some cases adopting the local indigenous knowledge) and
processes involved in the productive activities;
ƒ The way basic services are provided, managed and distributed to the population;
ƒ The way communication and information are provided; and
ƒ The way consumers purchase”

Whilst CP continues to be an important building block for SCP, UNEP has developed
complimentary SCP activities, including energy efficiency (both industrial (typically part
of CP) and non-industrial (e.g. buildings, not typically part of CP)), Multilateral
Environmental Agreements (MEAs), sustainable procurement, sustainable consumption,
Design for Sustainability (D4S) and Global Compact (GC).

3
1.2 Cleaner Production Programme

As summarised in section 1.1 the United Nations Conference on Environment and

Development (UNCED) in Rio in 1992 had provided in Agenda 21 [15] a clear mandate
to the international community to support developing countries and economies in
transition with capacity building and implementation of CP. UNIDO and UNEP had
already started to collaborate on specific projects, for example on a collaborative CP audit
manual [18], and then both launched in 1993 a comparable CP demonstration project
[19], respectively in India (UNIDO) [20] and China (UNEP with funding support from
the World Bank). Despite a number of differences, both projects were essentially
modelled on the Dutch PRISMA Project [7], and combined in-plant demonstrations, with
the development of manuals and policy analysis, and capacity building. Simultaneously,
several other donors funded bilaterally programmes on CP or related topics in developing
countries, for example the Environmental Pollution Prevention Programme (EP3) in
about a dozen countries (see [21] for a summary of key pioneering initiatives on CP in
developing and transition economies). The UNIDO and UNEP projects in India and
China were successful in demonstrating the potential for CP implementation in local
industries, and identified the need for some ongoing platform at the national level for
fostering CP uptake. Hence the notion of National Cleaner Production Centres (NCPC)
was conceived, which some resemblance to the Pollution Prevention Programmes in the
USA and CP-related centres in Europe.

The core idea was that NCPCs would be created within national host institutions, to
establish an entity that provides four types of CP services:

1. CP assessments/in-plant demonstrations: technical assistance provision to companies

and other organisations for the identification, evaluation and implementation of CP
opportunities;

2. Information dissemination and awareness raising: development and distribution of

promotional materials and delivery of awareness sessions or workshops to put CP
nationally on the agenda of government and the private sector;

3. Training: delivery of training programmes to establish a cadre of CP professionals

who could assist businesses and other organisations with CP implementation; and

4. Policy advice: liaison with government and other key stakeholders to identify ways to
create a policy environment more conducive to CP.

In this initial set up the NCPC was perceived as an entity that could on an ongoing basis
‘deliver CP demonstration projects’ in a manner that UNIDO and UNEP had just gained
experience with respectively in India and China. A ‘lean’ implementation model was
adopted where NCPCs would get some funding support to fund a Director and some
project activities, for a limited period of 3 to 5 years. The local host institution would then
provide in kind support, in principle through a Deputy Director and access to facilities
and services of the host institution. It was expected that such lean model would have more
chance to be locally sustainable in the longer run. Upon initial training, it was expected
that the NCPC would launch activities on its own, under the guidance and direction of an
International Reference Centre ((IRC) the term counterpart institution was initially used)
with would essentially ‘twin’ with a NCPC.

4
The Government of The Netherlands provided seed money to kick start the Programme,
and with further funding support from UNEP, Denmark and Austria the first NCPCs were
established in late 1994 in China, India, Tanzania, Zimbabwe, Mexico, Brazil (self
funded), Czech Republic and Slovakia. A second generation of NCPCs was established
from 1998 onward when the Governments of Switzerland and Austria provided funding
to set up NCPCs in Central America. A fifth core service area was added, namely support
for the identification, evaluation and transfer of ESTs. While some of UNIDO’s activities
in regard to EST transfer were from then on channelled through the UNIDO-UNEP CP
Programme, other EST initiatives were undertaken in isolation from this Programme (for
example the programmes on EST transfer for environmental remediation of the Danube
River and the Black Sea).

From 1998 onward the programme gradually expanded and now has activities in some 35
countries. About half of these (still) receive institutional funding through the UNIDO-
UNEP CP Programme, while the other half is strictly speaking independent from the
Programme, even though they may still be involved on a project basis. Figure 1.1 shows
the map of the geographical scope of the Programme in 2007.

Figure 1.1: Map of project locations in the UNIDO-UNEP CP Programme

(http://www.unido.org/doc/4450)

Though initially the management and programming of the activities for the NCPCs was
shared between UNIDO and UNEP, this changed by the late 1990’s as a result of multiple
changes in project staff, organisational support and organisational priorities within both
agencies. In the following period, UNEP had relatively little input to the development of
NCPCs and the overall strategic direction. UNEP worked on specific projects with
selected centres, for example with regard to energy efficiency and product design.
UNIDO maintained control over the institutional funding for establishment and operation
of NCPCs, and therefore controlled the bulk of the finances available to the Programme
and also providing a greater management contribution. Even though this programme
evaluation uses the term joint UNIDO-UNEP CP Programme this in no way implies that
both organisations have had, and/or will continue to have an equal role in programming,
management and administration of the programme.

5
Programme implementation is therefore currently achieved through multiple project
agreements, mostly on a one-on-one basis for a given period (initially three years) with a
donor and host country. In addition some multiple country projects have been
implemented, many of these under the auspices of UNEP (for example projects funded by
the Global Environment Fund (GEF) on ‘Energy Efficiency through Cleaner Production
and Environmental Management Systems’ (EECPEMS) and CP in Multilateral
Environmental Agreements (MEAs), multi-country projects funded by the Government of
Sweden on CP in Asia Pacific (including Greenhouse Emissions Reduction in Industries
in Asia Pacific (GERIAP), and a multi-country project funded by the Government of
Norway on CP financing). The UNIDO-UNEP CP Programme is thus in principle a
theoretical notion as there is no current, overarching programme strategy and
implementation agreement between UNIDO and UNEP. Most recently however there is a
genuine commitment at the highest levels in both organisations to strengthen coordination
and cooperation around the network of NCPCs. A tangible output from renewed
commitment is this programme evaluation, which also aimed to strengthen collaboration
in the UNIDO-UNEP CP Programme significantly.

The Programme has thus evolved as the sum of inputs, outputs and outcomes of a series
of similar but not identical projects with different timelines, scales, budgets, donors and
host institutions in different countries. Figure 1.2 provides a conceptual entity diagram for
the Programme. A distinction is made between institutional funding (on left hand side)
and project based funding (on right hand side) (6). However with the diversity of the
different NCPC projects, many variations exist.

Figure 1.2: Schematic entity diagram for the UNIDO-UNEP CP Programme

Project Donors
Institutional Donors
GEF, Swizerland,
Austria, Switzerland, UNIDO UNEP
Austria, Norway,
others
Sweden, others
Project
Funding
Centre UNIDO Cleaner UNEP DTIE
Funding Production and Sustainable
Contents
Energy Efficiency Consumption and
Branch Production Unit

Project Funding
Institutional Funding &
Financial Control
Cleaner
Production CP Projects
Internationall Consultants
Host Institutions
Reference Centres

Capacity
Institutional Building
Technical
Support
assistance

National Centres or
National Centres Other CP
Programme Units (Support
(Post Support Stage) Centres
Stage)

SERVICE DELIVERY TO NATIONAL STAKEHOLDERS

Country-Specific Centre Based Multi-Country Project Based

6
It should be pointed out that after an initial establishment period, in some countries, other donors or intergovernmental
financial institutions have become the source of institutional funding for the NCPC established through the UNIDO-UNEP
Programme. The distinction between institutional and project funding is therefore to a certain degree fluid. In the remainder
of this report, the term institutional funding will be used for funding provided through the UNIDO-UNEP Programme for
establishment and operation of the NCPC/NCPP and that is not, or not exclusively, linked to specific service delivery by
the respective NCPC/NCPP to its national customers.

6
As per Figure 1.2, in the context of this programme evaluation, the Programme is
understood to comprise of four components, respectively:

1. National Centres or Programmes (NCPCs/NCPPs): service delivery institutions

established in the host countries that deliver CP and CP-related services;

2. Programme Management: the activities of the CP Unit in UNIDO Headquarters in

charge of project administration, strategy development, liaison with donors, reporting
and financial control;

3. Technical Assistance: providing access to know-how, expertise and skills in CP and

related areas to the NCPCs, through training and provision of international experts;
and

4. Regional Networking: activities organised by UNIDO to achieve exchange of know-

how and experience between staff of the NCPCs/NCPPs in different countries, for
example through meetings of the directors, regional projects etc.

The NCPCs/NCPPs have highly similar features and activities in the different countries
(as summarised in particular in Chapters 3 and 4 of this report). Other CP like entities
have been established with comparable roles by other donors in other countries, in
relative isolation of the UNIDO-UNEP CP Programme. The Government of Denmark has
sponsored sector and policy specific CP projects for example in South Africa and
Vietnam, both countries with a NCPC under the UNIDO-UNEP CP Programme, and
Thailand and Malaysia, both countries without a NCPC. The Government of Germany
sponsored GTZ for the implementation of training and capacity building in profitable
environmental management – some of these activities have taken place in countries with
NCPCs (e.g. Egypt, India, Vietnam) and others in countries without a NCPC (e.g.
Indonesia and Thailand).

Outside the UNIDO-UNEP CP Programme, it would appear that the Regional Network of
the World Business Council for Sustainable Development (WBCSD) is currently the only
network with a comparable global spread. The WBCSD regional network is however
business driven and membership based, and includes activities on Eco-Efficiency as well
as other corporate sustainability topics (e.g. business for the poor, corporate social
responsibility, accountability and transparency). The Regional Network of the WBCSD
puts an emphasis on business self-initiative, awareness raising and business and policy
dialogues, and does not deliver services as in the case of the NCPCs (however in most of
the developing countries the national secretariat does have a capacity to undertake
project-based services to member companies). For information purposes, Figure 1.3
shows the geographic distribution of the Regional Network of the WBCSD. About half
the NCPCs/NCPPs are in countries where there is also a Regional Partner of the WBCSD.
In some countries the Regional Network and NCPC operate in relative isolation (for
example South Africa, China, India) whereas in other countries there is a direct link (e.g.
Regional Partner of the WBCSD being the host institution for the NCPC (notionally in
Mozambique and previously also in Zimbabwe)).

7
Figure 1.3: Regional Network of the World Business Council for Sustainable Development
(http://www.wbcsd.org/templates/TemplateWBCSD5/layout.asp?type=p&MenuId=NjM&doOpen
=1&ClickMenu=LeftMenu)

1.3 Independent Evaluation

This Independent Programme Evaluation for the UNIDO-UNEP CP Programme was

initiated to document and assess the activities and results of the NCPCs/NCPPs
established, taking the available programme documentation as a reference point. It was
also aimed to provide suggestions and recommendations for strengthening the global
network of NCPCs/NCPPs, for improving service delivery in the host countries and for
further catalysing sustainable industrial development in developing countries and
economies in transition.

Originally several related initiatives, in particular from bilateral donors, were meant to be
included in this evaluation project in order to broaden the scope of the evaluation and
ensure learning effect for CP implementation beyond UNIDO and UNEP. However, the
complexity of the programme evaluation finally allowed only for the inclusion on one
such initiative, by including two NCPCs funded bilaterally by the Government of
Switzerland in the independent evaluations (Colombia and Peru) and by reviewing
evaluation reports prepared on NCPCs/NCPPs.

1.3.1 Previous Evaluations

Even though this programme evaluation is unique in its scope and coverage, there have
been earlier evaluations at programme level, in particular:

ƒ 1996 Programme Evaluation [22]: This evaluation was undertaken some 2 years after
selection of the host countries for the first generation of NCPCs. Field visits were
undertaken by evaluators of the International Institute of Industrial Environmental
Economics (IIIEE) (of Lund University, Sweden) to China, India, Zimbabwe,
Slovakia and Czech Republic (five of the eight first NCPCs) to interview staff and
clients of the NCPCs. The evaluation found that the NCPC programme was relevant
and viable, but needed adjustment, in particular in regards to customising the NCPC
concept and its services’ portfolio to national circumstances, increasing transparency
in programme management, improving networking, and measurement of programme
success. It was also pointed out by the evaluators that a NCPC with a primarily

8
 information and networking function would not be a value proposition that could be
expected to become financially self-sustainable on the short to medium term.

ƒ UNIDO Programme Evaluations: in separate exercises UNIDO evaluated the

performance of the NCPCs in Central Europe (2002) and developing countries (1999)
[23]. The evaluation found that CP was a cost effective approach towards sustainable
development and that the CP methodology as being advocated by the NCPCs was an
effective tool for identification and prioritisation of technology changes that yield
both environmental and economic benefits. It was also found that dissemination and
application of the CP concept among small and medium sized enterprises on the basis
of its economic merits did not occur easily and needed support through promotional
activities and policy changes. The Programme was found to be most effective in
regards to awareness raising, training of CP assessors and introduction of CP in
university curricula and policy frameworks, whilst economic and environmental
benefits at industry level were rather modest compared to the potential existent in the
industry sector. The evaluators found that NCPCs should not be evaluated exclusively
or primarily by impact at the company level, but rather by the impact at the industry
level in terms of their success in transferring the CP concept and its tools to other
organisations/consultants and their contribution to the formulation of conducive CP
policy frameworks. On the basis of their findings they also postulated that the NCPC
programme would have better chances of achieving significant impact in countries
that have a larger segment of well performing industry with consolidated
management systems than in countries in which large segments of industry face
rudimentary problems of survival and are in need of restructuring and consolidating
management functions first.

ƒ NCPC Lessons Learned: in the lead up to the Johannesburg WSSD, UNEP prepared a
booklet of lessons learned from the NCPC programme [24]. It appears that the
booklet was largely based on information obtained from previous evaluations, and
experience of UNEP and UNIDO staff in working with the NCPCs. The guiding
messages are organised according to the start up phase, support phase and post-
support phase for funding under the NCPC programme. The messages argue for
targeted and focused service delivery, commitment to information dissemination and
liaison, outcome and result oriented service delivery, local implementation and
managing the tension between private and public interest role of the NCPC. Concern
is also expressed that emphasis on commercial service delivery drives the NCPC into
becoming a commercial service provider to large, creditworthy businesses.

ƒ SECO Impact Evaluation [25]: the Swiss government commissioned a separate

impact evaluation for seven NCPCs or alike Sustainable Enterprise Development
(SED) centres funded by the Government of Switzerland (respectively in Columbia,
Costa Rica, El Salvador, Guatemala, Morocco, Peru and Vietnam). Only direct
impacts from CP implementation in companies were considered. The evaluation
focused on quality of the CP assessment services and reports of the respective centres.
The report stated that there was better potential for impact from CP service delivery
in medium to large enterprises, and that group based approaches with follow up
implementation support should be considered.

The centrepiece of this impact evaluation was an estimate of the financial benefits
from CP assessments. These were estimated on the basis of determining the
percentage share of options implemented in a selection (but not necessarily a

9
 randomised selection of assessment reports) multiplied by the total savings identified
in each of the CP assessment reports, and kept constant for five years. A proxy cost
benefit ratio was then calculated on the basis of 1/3 of the Swiss donor contribution
(cost) versus the financial savings achieved over the five-year period in all companies
(benefit). In doing so, it was found that “every dollar invested by SECO had resulted
in 3.5 dollar saved in a participating company”.

Even though an impact evaluation is in principle to be applauded, this particular

SECO impact evaluation could not be endorsed by this programme evaluation, for a
number of inter-related reasons. Full project benefits (CP implementation) are related
against partial costs (only part of the costs of one of the project contributors), and the
methodology overestimates savings and underestimates costs. The country datasets
are also statistically unlikely. The limitations of this impact evaluation have been
reviewed in detail in the country evaluation report for Vietnam but they apply to all
countries covered by the impact evaluation.

The findings of these programme evaluations have influenced the overall direction and
administration of the UNIDO-UNEP CP Programme. This is covered in the results of the
programme documents’ review in Chapter 2 of this report.

In addition to these programme evaluations there have been several project level
evaluations as part of the funding cycles for most, but not all, of the NCPCs. All except
one (Sri Lanka) of such country level evaluations were performed by international
evaluators assisted with national consultants [26]. The Sri Lankan experience showed that
rigorous and independent evaluation of a NCPC does not have to depend on international
consultants. The country level project evaluations are however not reviewed here in any
further detail. Instead, the independent country level evaluations undertaken for the
selected NCPCs/NCPPs for this programme evaluation cover key findings from any
project evaluations that have been undertaken in the respective countries (see chapter 4,
and annex II to this main report).

1.3.2 Evaluation Methodology

This global programme evaluation was structured around four primary and two secondary
evaluation criteria. The primary criteria relate to the uptake of CP, and are:

1. Relevance: are the elements of the programme (i.e. the CP concept, the CP services,
the NCPC institution, the global network and the technical assistance inputs)
applicable and valuable for the intended beneficiaries (i.e. the private sector,
government, academia and research institutes in the host country)? ;

2. Effectiveness: does the design of the programme (i.e. national centres, global
management and networking, and technical assistance) and its implementation enable
the Centres and beneficiaries to achieve the programme’s intended results (i.e. uptake
of CP)?;

3. Efficiency: is the programme designed and implemented to achieve optimal benefit

from its available resources? Are the Centres and other programme activities
managed and administered in a manner that fosters service delivery to beneficiaries?;
and

10
4. Sustainability: is it probable or likely that the benefits (e.g. availability of CP
services, environmental and productivity benefits in industry, etc) achieved from the
programme will continue into the future?

The secondary criteria assess the success of the CP Programme as a development

assistance intervention. These represent two elements of best practice for project
execution and management, and are therefore highlighted separately. These cover:

5. Capacity Development: does the programme develop essential capacities (e.g. in

regards to resource productivity, environmental management, entrepreneurship,
and/or public private partnerships) for local stakeholders to improve their current and
future well-being?; and

6. Ownership: do local stakeholders regard the programme as their own and do they
make commitments to advance the programme’s aims and objectives and act on its
outputs?

To a certain extent the primary criteria are hierarchical and sequential, as a reasonable
degree of relevance is required to achieve some effectiveness, and effectiveness is
conditional for both efficiency and sustainability. There is also some overlap between the
secondary and primary evaluation criteria. Capacity development is mostly related to
effectiveness and efficiency. Ownership on the other hand is principally influenced by
relevance and sustainability. This inter-relatedness of the evaluation criteria is displayed
in Figure 1.4.

Figure 1.4: Evaluation criteria for the programme evaluation

SECONDARY 5 SECONDARY 6
SECONDARY 5

CAPACITY
CAPACITY
DEVELOPMENT IMPACT OWNERSHIP
DEVELOPMENT
DEVELOPMENT

INCREASED UPTAKE OF
CLEANER PRODUCTION PRIMARY 1 PRIMARY 2 PRIMARY 3 PRIMARY 4

RELEVANCE EFFECTIVENESS EFFICIENCY SUSTAINABILITY

The evaluation was performed by an international expert team comprising of three

independent consultants and one programme officer from UNIDO’s Evaluation
Group/Bureau for Organisational Learning. It was overseen by a Steering Committee with
representatives of UNIDO and UNEP (both the programme units involved, as well as the
respective evaluation units) and the Governments of Switzerland and Austria. Norway
and Germany participated partially as observers. The evaluation kicked off in April 2007,
and substantive interim findings were presented to the Meeting of the Directors of the
NCPCs/NCPPs, held in Semmering (Austria) from 24-26 September 2007. This report
(including the annexed country evaluation reports) have been finalised thereafter taking
into consideration the valuable feed back received during and after the Semmering
meeting.

The evaluation methodology is displayed in Figure 1.5. There are three main ‘pillars’ on
which this global programme evaluation is based, respectively: programme review, self
evaluation and independent evaluations.

11
 Figure 1.5: Overview of the evaluation methodology

Independent
Self Evaluation of
Programme Country
Centres
Review Evaluations
(n = 38)
(n = 18)

Country Profiles
Evaluation
Reports

Portfolio Analysis

Programme Evaluation

Conclusions &
Recommendations

1. Programme review: a review of developments in the Programme over time, with

particular emphasis on programme strategy, management and administration, and
adaptive management and learning over time. This review is largely based on a
review of various strategies, business plans and reports produced by the CP Unit in
UNIDO in charge of the day-to-day management and administration of the UNIDO-
UNEP CP Programme.

2. Self Evaluations (by Centre Directors): a comparative analysis of the experiences and
views of the directors of the NCPCs/NCPPs, as expressed by them in response to two
email-administered questionnaires. The first survey covered characteristics of the
NCPC/NCPP (e.g. its institutional set up, budget, staffing, activities) and an
assessment of the performance of their NCPC/NCPP against five evaluation criteria
(relevance, effectiveness, efficiency, sustainability and ownership). The follow up
survey explored in greater detail the level of interest and involvement in various CP
and CP-related service areas (such as sustainable procurement, occupation health and
safety, Multilateral Environmental Agreements, etc.)). This self assessment covered
all NCPCs and NCPPs (total of 38) in the UNIDO-UNEP CP Programme.

3. Independent Country Evaluations (by evaluation team): members of the international

evaluation team visited a selection of the participating NCPCs for an independent
review and assessment of the results and experiences of the NCPC from a programme
level perspective. These independent evaluations were based on document reviews
and discussions with NCPC staff, board members, clients and relevant government
and industry representatives. One member of the international team spent between 2
and 5 working days in the country and was assisted by a national consultant who was
independent from the NCPC. The selection of countries to be visited was made by the
Steering Committee at the suggestion of the international evaluation team. In doing
so, it was attempted to arrive at an illustrative selection, including NCPCs in different

12
 stages of establishment and funding, with different types of host institutions, with
different donors and in different parts of the world. Visits were undertaken to 19
countries, between April and September 2007. These were: China, Colombia, Costa
Rica, Croatia, Egypt, El Salvador, Guatemala, India, Kenya, Mexico, Morocco,
Mozambique, Nicaragua, Peru, Slovakia, Sri Lanka, South Africa, Uzbekistan and
Vietnam. For 18 countries detailed country evaluation reports were prepared.
Slovakia was excluded as it turned out that the Slovak NCPC no longer fulfils a
public role in advocating CP to industry and government (albeit remaining active as
commercial CP service provider to the UNIDO-UNEP CP Programme and other
clients).

These three components provide the factual information (or ‘evidence base’) for the
independent evaluation. The analysis and evaluation was also divided in three constituent
parts, respectively:

1. Portfolio Analysis: an analysis of the status quo of the UNIDO-UNEP CP Programme

in the participating countries, on the basis of a set of discriminating factors in the
establishment, operation, management and governance of the NCPCs/NCPPs in the
programme countries. The aim was to find common trends in development of
NCPCs/NCPPs and potentially clusters of common activities or areas of common
need in groups of NCPCs.

2. Programme Assessment: an overall assessment of the UNIDO-UNEP CP Programme

against the primary and secondary evaluation criteria, i.e. relevance, effectiveness,
efficiency, sustainability, capacity building and ownership.

3. Conclusions and Recommendations: an integrated set of conclusions from the

independent evaluation and associated recommendations organised in clusters, that
each provide a lever for improvement of the Programme.

The evaluation was executed between April and December 2007. Interim results
including draft conclusions and recommendations were presented for review to the 9th
Annual Meeting of NCPC Directors, held in Semmering (Austria) on 24-26 September
2007. A comprehensive draft was released in January 2008, and was finalised in April
2008 taking into consideration the comments and suggestions from UNIDO, UNEP and
donor representatives.

The evaluation faced several practical limitations: documentation was mainly available at
the project/country level, not at the programme level; no financial figures at programme
level were made available to the evaluation team; for some of the programme documents
analysed the respective period of validity was not clear; and due to the long period
covered not all people involved in design and implementation could be consulted.

1.4 Report Overview

The remainder of this main evaluation report is structured in three main parts and six
chapters.

13
Part I (Findings) constitutes the evidence basis for this programme evaluation. The three
chapters each cover one of the main information ‘pillars’, respectively findings from the
programme review (Chapter 2), findings from the self evaluation (Chapter 3) and findings
from the independent country evaluations (Chapter 4).

Part II (Analysis and Assessment) provides for an analysis and assessment of the
Programme, integrated from the findings of the three sources of findings. Chapter 5
(Portfolio Analysis) focuses on the current status of the Programme and attempts to
highlight parallels and synergies between activities of NCPCs/NCPPs in different
countries with varying degrees of industrial development and national socio-economic
and environmental priorities and objectives. Chapter 6 (Evaluation) contains the
programme level assessment by the evaluation team of the performance of the UNIDO-
UNEP CP Programme against the primary and secondary evaluation criteria, as well as an
overall summary assessment of the Programme’s main achievements.

Part III (Conclusions and Recommendations) provides a comprehensive set of

conclusions and recommendations (Chapter 7).

This main report is accompanied by two sets of contributing reports that will be made
available on request by the UNIDO Evaluation Group. The first set contains country
profiles for all NCPCs/NCPPs. These have been based on the survey responses from the
respective directors. The second set contains the independent evaluation reports for the 18
visited countries by the international consultants. These are comprehensive reviews of the
status of development and achievement of the respective NCPC by the respective
evaluator who visited the country and contain specific conclusions and recommendations
at the national level.

14
Part I:

Findings

15
2
Programme Review
_______________________________________

2.1 Overview

This chapter looks at the design, implementation and the results of the programme level
activities in the UNIDO-UNEP CP Programme. As noted before, in the absence of an
overarching project strategy and programmatic funding, the Programme evolved over
time as a set of projects. Also roles and responsibilities were not equally shared, with
UNIDO having the lead role in programming, implementation and ongoing review.

The programme review presented here therefore had to take as the basis the explicit and
implicit objectives of the UNIDO-UNEP CP Programme. It then analysed the activities
undertaken by UNIDO and UNEP to achieve these objectives, including the cooperation
and coordination with donors (especially the current main donors Austria and
Switzerland).

The need to include implicit objectives (i.e. objectives not formulated explicitly in a
programme document) into the analysis of programme design arises from the fact that
there is not a single, comprehensive programme document that would provide
information with regard to the scope, actors and objectives of the Programme (see 2.2.
below). The programme design is first reviewed (section 2.2), followed by reviews of
programme implementation (section 2.3) and programme level results (section 2.3). The
final section contains the key findings (section 2.5).

2.2 Programme Design

2.2.1 Programme Concept

The basis for the design of the UNIDO-UNEP CP programme is the concept of Cleaner
Production (CP) itself (see section 1.2). The CP adopted in the initial programme
document is consistent with the consensus definition arrived at in the mid 1990’s: “CP is
the continuous application of an integrated preventive environmental strategy to
processes, products and services to increase eco-efficiency and to reduce risks to humans
and the environment”[9]

This definition explicitly includes life cycle approaches for products. However, early
programme documents point out that the emphasis of programme activities is on the
production process and not so much on products. The concept of energy efficiency is also
covered by the CP definition, even though it is not explicitly singled out as one of the
applications of CP.

17
The above core definition of CP has been maintained as the centrepiece of the UNIDO-
UNEP CP Programme until today. This is consistent with the use of CP in the
international community. While UNIDO, given its mandate to promote industrial
production processes, has focused its activities within the core concept’s scope, UNEP
has widened its own programme to include consumption issues into ‘Sustainable
Consumption and Production (SCP’ programme (see also section 1.1), in response to the
Johannesburg Plan of Implementation of the World Summit on Sustainable Development
(WSSD). Furthermore, there is an increasing tendency to use the NCPCs/NCPPs as local
partners for the implementation of donor programmes in areas related to, but not
necessarily part of the core concept of CP (in particular Corporate Social Responsibility
(CSR) (or social entrepreneurship), and implementation of Multilateral Environmental
Agreements (MEAs), including their financing mechanisms, like Clean Development
Mechanism (CDM) and transfer of Environmentally Sound Technologies (ESTs)).
Remarkably however, UNIDO has implemented its own programmes on CSR and EST
transfer at arm’s length of the UNIDO-UNEP CP Programme (e.g. the projects on EST
transfer for environmental amelioration of the Danube and Black Sea and REAP
(Responsible Entrepreneurs Achievement Programme
(www.unido.org/index.php?id=o42159).

It can already be pointed out here that this evaluation found ample evidence that the
relevance of the CP concept has been high throughout the implementation period.
International developments such as trade liberalisation, increasing energy prices,
advances in environmental legislation, all tend to increase the relevance even further. The
extent to which these developments have influenced the performance of particular
NCPCs/NCPPs and the impact of their activities depends heavily on the specific
framework conditions (legislation, enforcement, resource prices, etc.) in the different
countries and hence there continues to be variability in the relevance of the CP concept, in
particular for the private sector.

2.2.2 Programme Strategy and Objectives

The UNIDO-UNEP CP Programme was designed as a capacity building programme. It

has been applying an ‘indirect’ approach to introducing CP in developing countries by
first creating local capacities that are then for a defined period of time supported and
further strengthened. These can then be utilised for implementation of national and
international interventions. The specific approach to capacity building was to the set up
new entities, the NCPCs, most often created as (semi-) autonomous centres, within
existing host institutions or creating completely new institutions.

The original programme strategy had an almost exclusive focus on the establishment of
NCPCs. The programme was then referred to as the UNIDO-UNEP ‘NCPC Programme’.
Later both organisations developed documents that referred to a ‘CP Programme’,
indicating a wider scope of the Programme, including interventions other than
establishing and supporting NCPCs.

Based on the experience from the set up of the first sets of NCPCs, so-called National CP
Programmes (NCPPs) were defined as alternatives to NCPCs. NCPPs were applied in
countries where some demand for CP promotion existed but no sufficient capacity was
available for the establishment of a NCPC. The activities of NCPPs resemble to a large
extent those of NCPCs (demonstration projects, training, information dissemination) and
in some cases are designed to ‘prepare the terrain’ for the set up of a NCPC later on. This

18
deviation from the exclusive focus on NCPCs indicates that the programme management
started considering models other than NCPCs to promote CP at the country level. The
NCPP concept, however, was applied only in a small number of cases. Of the 34 NCPCs
and NCPPs listed in 2007 on the UNIDO web page only Armenia and Lao PDR are
officially registered as NCPP (Bulgaria, Romania, Ukraine and Montenegro do have
ongoing NCPPs but these are not listed on the webpage).

As NCPCs in different countries matured, the need for general capacity building support
to those NCPCs diminished. As a result, over time some elements were introduced to
reflect a wider approach of CP promotion. This includes regional networking initiatives
(such as the CP-LatinNet network in Latin America) and, more recently, also specific
technical initiatives such as Chemical Leasing (CL) and SAICM (Strategic Approach to
International Chemicals Management). Arguably the Latin American experience could, if
proven successful, become a model for replication globally. In the other three key regions
for the programme however there are reasonably active Regional Roundtables, that could
be better utilised by the Programme as a means for regional networking (respectively in
Asia Pacific, Africa and Europe).

As early as 1997 plans were made to create a global CP network that would be open not
only to NCPCs supported by UNIDO but also to other CP-related institutions [27]. This
plan has been reiterated throughout the years (see for example the Mayrhofen CP
Programme Declaration from 2003). However, with exception of the Latin American
regional network, the networking activities of the programme have until now not been
developed in a systematic way.

This might be related to the fact that no clear strategy, objectives and outcomes have been
defined for the global network. Instead, the definition of the network always started at the
activity and output levels, listing the possible lines of activities and outputs without
clearly explaining what the ultimate aims of these activities would be. Obviously this has
also limited the possibilities to assess the potential effectiveness of a NCPC/NCPP
network vis-à-vis other possible interventions (e.g. establishing a global information
centre for CP) and the possible complementary nature of the UNIDO-UNEP network in
relation to other networking initiatives (e.g. the regional roundtables on SCP, GTZ
network on profitable environmental management, and regional network of the WBCSD,
etc). Furthermore, no additional resources were made available for global networking.
Given the very limited staff and funding resources of the UNIDO-UNEP CP Programme
pro-active facilitation and support for networking could not be provided in parallel with
the ongoing activities for the set-up of new and the support to existing NCPCs/NCPPs.

More has been achieved in the area of regional networking. The formation of the “CP
LatinNet”, a networking initiative for the Latin American NCPCs (see box 2.1),
overcomes most of the barriers described above. Separate resources were mobilised and
an effort was made to establish clear goals for the network.

Apart from the common definition of the CP concept and a generic Letter of Agreement
(LoA) [28] on interagency co-operation (see below) and despite of the frequent references
to the joint UNIDO-UNEP CP Programme, at the time of evaluation in 2007 there was no
joint document spanning the activities of both agencies for CP promotion in general or the
management of the UNIDO-UNEP CP Programme specifically.

19
Objectives:
The overall objectives of the UNIDO-UNEP CP Programme are referred to in numerous
documents, the most important of which are:
x The programme documents for the support for the establishment of the first batch
of NCPCs in 1994 (the ‘old’ NCPCs’) [29];
x The information brochure of the UNIDO-UNEP NCPC Programme [30];
x The UNIDO Holistic and Sectoral CP strategy 2003 -2006 [31];
x The UNIDO CP Programme Business Plan 2003 – 2005 [32]; and
x The UNIDO NCPC webpage (www.unido.org/index.php?id=o5133).

The following development objectives have been extracted from these programme
documents. They provide testimony for the wide scope of the objectives that have been
formulated over the years:
x Reductions in risk to human health and the environment;
x Enhanced industrial productivity;
x Increased application of CP in the industrial sector;
x Incorporation of CP in the national environmental policy and legislation;
x Transfer of CP information and CP technology from developed and developing
countries to industrial enterprises and environmental management agencies in
(other) developing countries or economies in transition; and
x Economy-wide productivity gains for sustainable economic and social
development.

The ‘Holistic CP strategy’ [31] issued by UNIDO only, introduced further objectives,
among them the Millennium Development Goals (MDGs) 1 (poverty alleviation), 7
(sustainable development) and 8 (global partnership) (see also section 1.1). It also refers
to improved international market access of companies in developing countries.

UNIDO’s business plan 2003-2005 for the CP Unit [32] established a number of
‘strategic objectives’ for managing the UNIDO-UNEP CP Programme, such as:
x Strengthening the network of NCPCs and NCPPs;
x Fostering international business cooperation and investments in ESTs;
x Integration between CP and other tools (e.g. life cycle assessment);
x Promoting NCPCs and NCPPs as partners for the implementation of MEAs; and
x Fostering and establishing regional networking.
These objectives, and indeed the business plan, define the scope and activities of the CP
programme management unit in UNIDO instead of the objectives for the UNIDO-UNEP
CP Programme itself.

Currently the UNIDO web-page lists another set of objectives of the NCPC programme
(www.unido.org/index.php?id=o5133):
x Increase competitiveness;
x Open access to new markets;
x Stimulate public-private partnerships; and
x Promote CP investments and CP technology development and transfer.

A review of these formulations of objectives shows that some are rather outputs (e.g. the
establishment of regional network of NCPCs, integration of LCA into CP), and others are
outcomes (incorporation of CP in legislation) or impacts (productivity gains). Some are
also simply activities (strengthening the network, promoting NCPCs as partners for

20
MEAs). The formulation of objectives reflects that strategy documents have defined the
objectives and activities of the programme management unit without clarifying the
Programme itself, which in turn presents a barrier to more results-oriented programme
management.

More importantly, there is no distinction between those objectives that constitute the
Programme’s development objectives and those that are not directly related to the
Programme, but where positive contributions can be expected depending on specific
applications. The main issue in this context appears to be the question whether poverty
alleviation should be among the direct objectives of the UNIDO-UNEP CP Programme or
not. Similar to what will be discussed below in regard to ‘outcomes’, in many instances a
trade-off between poverty alleviation and reduction of environmental impact can exist.
Cleaning up a pollution hotspot might require measures that do not directly (i.e. at least
not in the short term) alleviate poverty (e.g. the preventive CP approach promotes the
introduction of more efficient technologies which in turn might be less labour intensive
and lead to loss of employment opportunities for poor families).

It is not argued here that poverty alleviation cannot be a direct objective of the UNIDO-
UNEP CP Programme. However if it is, there should be a clear understanding what is
being understood as poverty alleviation (e.g. better working conditions, better
environmental health conditions in poor communities, or narrowly speaking only more
jobs or higher incomes). Likewise not all of the Programme’s interventions will
contribute to poverty alleviation, productivity gains and environmental impact reduction
at the same time and to the same extent. So when a component or project is designed to
achieve the Programme’s goals, it should be explicitly and clearly stated which of the
Programme’s goals will be aimed at and a logical means-end relationship between the
objectives and the planned outputs and activities should be established (7).

Outcomes
The concept of CP implies that a programme for its promotion would contribute to uptake
of CP practices, technologies and policies (outcome) with two parallel lines of benefits or
impacts: reduced environmental impacts of industrial activities (including processes and
products) and increased productivity of industrial activities (less resources used for same
output or same resources used for higher output).

From an analysis of documents from the early phases of the Programme it would appear
that at that time the focus of expected outcomes was clearly on ‘reductions in risk to
human health and the environment’. Apparently less importance was assigned to ‘to
enhance industrial productivity’ [29], even though the CP definition used listed eco-
efficiency (combined economic and ecological efficiency) as its first aim. It is however
noted, that there was always a strong emphasis on cost-efficiency of CP options (i.e. on
options that had a reasonably short pay back time in light of local environmental
standards and their status of enforcement).

Looking at more recent documents, it appears that over the years the emphasis has shifted
from the first to the latter main benefit. For example, the Business Plan 2003 – 2005 for
the UNIDO CP Unit [32] defines the mission of the CP Programme as follows: “assist
the national industries in improving their productivity and competitiveness to facilitate

7
For example, the activities and outputs needed for a CP intervention that aims at poverty reduction might be very different
from what is required for an intervention that focuses primarily on productivity gains or reduced environmental impact.

21
the access to new and more demanding markets through the diffusion of quality and
productivity enhancing ESTs, following a holistic and sectoral CP approach”. The
document however falls short in defining specifically what is understood as ‘holistic’ and
‘sectoral’, and how this would be different or superior to other CP approaches.

This apparent shift in emphasis is probably related to the evolution of the institutional
model of NCPCs/NCPPs towards financially independent service providers, who
naturally depend on their good relations with client companies and the private sector in
general. For the implementation of CP assessments in companies the productivity
argument is certainly the better entry point, unless there are urgent issues in regard to
enforcement and compliance with environmental legislation.

By and large the Programme appeared to have struggled to come to terms with the
existence of these two seemingly equally important outcomes, trying to maintain the
concept of a “win-win” situation, in which it is possible to achieve both benefits at the
same time. It might be argued that this can be realistically expected only when
environmental standards and legislation is being implemented and enforced and resource
prices reflect environmental costs to some extent. In the absence of a cost to non-
compliance or a reward for voluntary compliance or eventually beyond-compliance (e.g.
improved market access through a recognised eco-label or buyer requirements), the ‘win-
win’ premise is limited mainly to the implementation of ‘good housekeeping’ and other
no or low cost CP options (as reflected in the lower levels of implementation of higher
cost options reported by the visited NCPCs for the independent country evaluations (see
Chapter 4)).

An emphasis on the reduction of environmental impact would probably imply a focus on:
enforcement of existing, and where needed development of new, more stringent,
environmental legislation; capacity building in the public sector; and more proactive
targeting of sectors with significant environmental impact or most affected regions
(pollution black spots). It would define public policies and maybe even raise awareness
among civil society as to what can be expected from companies in terms of CP (Best
Available Technologies/Best Environmental Practices).

The emphasis on competitiveness/productivity gains, i.e. the benefits for the enterprises
concerned, implies being (private-) demand-driven with limited involvement in the
promotion of enforcement and stricter environmental legislation, implementation of only
the economically attractive (profitable) solutions. This in turn means that many solutions
that would in principle be economically viable under existing environmental legislation
(i.e. reasonable pay-back time, low risk) remain insufficiently attractive to warrant
investment.

Both approaches have their pros and cons. Which one is the better approach for the CP
Programme in a specific country or region depends on the local context and the priorities
defined by stakeholders. This diversity is reflected in the different orientations of the
NCPCs visited for this Programme evaluation (see Chapter 4). However, it is not
reflected in the design of the Programme and the projects to support the establishment and
strengthening of NCPCs, which in general assign a rather standardised role to a NCPC.

Outputs
The programme has produced outputs at the programme- and country levels. At the
programme level the development of the NCPC model can be regarded the main output,

22
while the establishment of individual NCPCs is the core output at the country level. The
NCPCs themselves then have also produced outputs (e.g. delivered training, undertaken
CP assessments), leading to outcomes (capacity built, and CP options being implemented)
and impacts (reduced environmental impact and improved resource productivity).

The establishment of a NCPC requires considerably more effort than the mere conduction
of training programmes or the implementation of demonstration projects. It implies a
long-term co-operation for institution building and requires continued efforts to create
local ownership and commitment for sustainability of the CP concept and the NCPC as
service providing institution.

While no evidence has been presented as to the logical design process that lead to the
conclusion that establishment of NCPCs would be the most effective, efficient and
sustainable way to achieve programme level objectives, the long-term approach implicit
in the establishment of NCPCs appears to be warranted in most cases. However, it should
also be noted that in some cases direct support to a number of existing institutions with
ongoing activities in CP and with sufficient capacities for CP uptake might have been
more effective, and should at least have been considered as an alternative to establishing a
new NCPC. Most importantly, the NCPC model developed at the outset of the
programme remained largely unchanged since then and continues to be a ‘one-size-fits-
all’ model, with no a-priori differentiation in services or institutional nature according to
the very different needs and framework conditions found in countries as different as India
and Nicaragua (see also paragraph 2.2.4). Other important outputs at the programme level
are: networking of NCPCs; technical assistance through a pool of International Reference
Centres (IRCs); fund raising; information and training materials; and monitoring &
evaluation (see section 2.3.)

2.2.3 Rationale and Logical Framework

There are many ways to conceptualise the rationale of the UNIDO-UNEP CP

Programme. One possible way is to use the concept of a logical framework, establishing
thereby an idealised means-end relationship between the programme’s objectives and
outcomes on one side and the different outputs and activities at programme and country
level on the other. The framework can then be used as a mechanism for systematic and
periodic consultations among programme stakeholders, especially UNIDO and UNEP.
The UNIDO-UNEP CP Programme has itself not yet produced such a logical framework.

For the purpose of analysing the Programme’s internal coherence, i.e. in how far the
above-mentioned logical and consistent means-end relationship exists, the evaluation
team undertook an attempt to re-construct a logical framework of the current Programme.
As a result of the analysis of the different strategy documents, including UNIDO and
UNEP initiatives, the overall programme logic found by the evaluation team is described
in Figure 2.1. A full logical framework would require the definition of indicators, means
of verification for such indicators and critical assumptions or risks that need to be
observed in order to ensure timely response to changed circumstances.

The programme management of UNIDO, in close cooperation with donors, made

considerable efforts to develop a set of meaningful indicators for monitoring of outcomes
at country level, in particular the projected environmental and economic benefits
achievable from in plant demonstrations. Less emphasis was on indicators at the

23
programme level and at the country level for institutional capacity development and
ownership. Risk management, i.e. the continuous monitoring and observation of a set of
critical assumptions and/or potential risks to the programme’s success in achieving the
objectives, has not been systematically undertaken in the programme. This is true for
both, country and programme levels. An example for risk management would be the
continuous monitoring of some international tendencies that influence the relevance and
effectiveness of CP, such as international resource prices, trade liberalisation and
environmental barriers to trade, etc.

Figure 2.1: Re-constructed Logical Framework of the UNIDO-UNEP CP Programme

Programme level interventions Country level Target groups
interventions
Impact/Objectives Sustainable industrial development National
economies or
sectors
Increased productivity of the private sector in developing Private sector in
countries and international economic integration target countries,
Global
Global and local environmental benefits (e.g. climate The environment
change related, environmental health) in target
countries,
population at
large in target
countries,
Global

contribution

Outcomes/immediate Reduced environmental impact of industrial activities The

objectives (“cleaner” production) environment
and population
affected by
pollution of
client
enterprises
Increased productivity and enhanced competitiveness (incl. Client
int. market access) enterprises of
NCPCs
Sufficient institutional capacity to support introduction of Public and
CP, improved enabling environment private
institutions
Ownership of the CP concept All target
groups

attribution

Programme level Country level

interventions interventions

24
Outputs x Project design and xInstitutional
initial support to set capacity created
up of NCPCs and through
NCPPs (“NCPC individual Enterprises
Model”) NCPCs
x Funds mobilization xDemonstration Public and
x Operational support, Programme
cases of CP, private
monitoring and level awareness institutions
quality control activities xCritical mass of
x Technical assistance support and professionals Affected
x Networking strengthen trained population
Activities the country xPolicies,
level and
x Manuals and create
incentives (e.g.
Guidelines synergies awards, tax),
x Promotion, etc.
Information sharing xInformation on
CP readily
available

2.2.4 NCPCs and Core Services

As mentioned above, very early in the process of programme design it was decided that
there was a need to establish ‘National Cleaner Production Centres’ (NCPCs) as the
principal output of the programme. The NCPCs were designed as vehicles to deliver a set
of services (activities). The services were adapted from earlier CP demonstration projects
(see also Section 1.2) in the expectation that Programme level objectives could then be
achieved. The initial four core services were [29]:
x Information dissemination;
x In-plant demonstrations/cleaner production assessments;
x Training and capacity building; and
x Policy advice.
In the second generation NCPCs (established after 1998) a fifth core service has been
added, namely:
x Transfer of Environmentally Sound Technologies (EST).

These services constituted the backbone of the NCPC model. They were applied in all
cases and the evaluation team found them even in the bilaterally managed NCPCs that
were established outside the UNIDO-UNEP CP Programme. The scope of these services
includes some that can be offered on a commercial basis with potential for private
benefits (at least under certain circumstances) and others that are of public nature and
therefore need public funding support. It is this broad scope that makes the institutional
model of the NCPCs suitable for countries or regions where no capacity and no
specialised institutions exist to deliver some of these services. The model seems less
adequate for countries or regions where considerable capacity exists for some of these
services in different institutions (for example in university departments,
research/technical institutes or business service providers (management consultants,
engineering design firms, etc.). In such cases either a CP Programme providing direct

25
support to existing institutions or an alternative model for a networked CP centre might
have been more effective. In the latter case the CP centre (or maybe a centre with a
different name) could be more of a niche player in one or more of the service areas. An
example for such a niche strategy could be a technology reference centre that specialises
in technology information and assessment (see the further discussion on centre models
and services in the portfolio analysis, in particular sections 5.3 and 5.4).

The design of the NCPC model does not include a description of the institutional nature
of the NCPC, in particular with regard to its relationship to other institutions. Such a
description would require the conceptual design of a number of possible models,
depending on the institutional, environmental and economic framework conditions for
CP.

2.3 Programme Implementation

This Section looks at the different aspects of the implementation of the CP programme,
starting with a general view on the programme management and the support given to the
set-up and operation of NCPCs. Then the aspects of networking, technical assistance and
information management are discussed.

2.3.1 Programme Management

Programme management in UNIDO is the responsibility of the Cleaner Production Unit

(CPU). The unit has four professional officers and one unit chief. Over the
implementation period the position of unit chief has been occupied by four different
individuals (and has been vacant from September 2007 to May 2008). One of the present
professional staff of the unit has been involved in the Programme from 1998, while other
professional staff has joined the CP unit more recently.

At UNEP programme management is with the Division of Technology, Industry and

Economics (DTIE), based in Paris. DTIE’s launched its CP Programme in 1990, in
partnership with many organizations including OECD, EU, UNIDO, and the World Bank
[33] (see also section 1.1). At the outset of the Programme the cooperation and division of
labour between UNIDO and UNEP was described in a Letter of Agreement (LOA) in
which UNIDO was assigned the role of ‘executing agency’, whereas UNEP was the
‘cooperating agency’. The executing agency (UNIDO) would then take the lead in setting
up 20 NCPCs in two phases, while the cooperating agency (UNEP) would provide
professional support in terms of methodologies and information (see also paragraph
2.3.1.2.).

It is noteworthy that the first phase of the CP Programme applied a strong programmatic
approach to the establishment of NCPCs. For the set-up of the first five NCPCs a project
document with common objectives existed. Since the project was funded through UNIDO
by a single donor (Government of The Netherlands, with co-funding by UNEP), the
negative effects of patchwork funding that later on affected the programmatic character of
the UNIDO-UNEP CP Programme did not exist during this early stages.

Another feature of the programmatic approach was the ‘solicitation and application
process’. In the first phase I 39 Institutions from 25 countries [27] countries participated
by submitting a proposal for establishing a NCPC (including administrative and technical

26
approaches and a budget) along the guidelines defined by UNIDO and UNEP. Among the
applications, UNIDO and UNEP first selected the countries with the best perceived
potential for CP and assimilative capacity for hosting a NCPC. This was followed by a
competitive selection process within the countries to select the most capable host
institution. This process shows that in the early phase programme level activities were
more pronounced and a clear strategy was developed and implemented.

In the later phases of the programme (i.e. approximately from 1998 onwards), these
programme level activities became less important and a more opportunistic approach to
establish NCPCs was followed. Attempts were made to ‘upscale’ the NCPC model, by
developing tools for countries to establish NCPCs by themselves [34]. However,
available staff resources of the UNIDO CP Unit were largely used to establish new
NCPCs in the same way as in the early phase (i.e. with direct involvement of UNIDO
staff in the NCPC management) wherever a request from recipient countries and donor
funds were available.

The UNIDO CP Programme management faced a number of internal systemic constraints

within UNIDO, which made pro-active programme development and effective thematic
leadership difficult:

x Starting in the early 90’s UNIDO had to downscale substantially its staff after the
withdrawal of Canada, the USA and Australia from the Organisation. This resulted in
increased pressure on remaining staff with less time for forward-looking activities
like programme development and strategic planning.

x Not only in the case of the CP Programme, but the funding of UNIDO’s technical
cooperation in general, has been to a large extent on a project-by-project and not on a
programme basis. This makes programme management more difficult, even if well-
qualified staff is available.

x CP as a concept lends itself for being promoted and advocated by an ‘epistemic

community’” (i.e. a network of knowledge-based experts) [35] UNIDO’s dependence
and/or focus on funding for technical assistance on a project-by-project basis allowed
little room for programme-level activities (such as research, expert group meetings,
etc.) and thus did not facilitate the emergence of such a group of experts with
oversight over, and/or influence on, the state-of-the-art in development and
implementation of CP concepts, methods, technologies and policies.

x UNIDO has a political mandate to be active in all its member countries. Requests for
cooperation from the countries are taken as a basis for the development of technical
cooperation activities. Management and staff therefore are bound to respond to such
requests which precludes the possibility of strategic selection of countries or regions
and for more innovative CP pilot activities.

x UNIDO implements technical cooperation with the agency execution model (8). At
the same time country support capacities of UNIDO (through country offices or

8
“Agency execution entails management by UN Agencies where activities require technical sector expertise or specific
management capacity and access to international networks; the government lacks the required management or substantive
capacity; or the parties prefer agency execution for other reasons. Under agency execution, the UN Agency may plan and
carry out the programme or project activities applying its own procurement procedures.” (definition from UNDP website).
Alternatives to agency execution are National Execution (NEX), Direct Execution (DEX) and NGO execution.

27
 cooperation with UNDP) are in many cases very limited. This usually requires from
HQ project managers a close involvement in operational issues (‘micro-
management’), draining valuable staff resources away from issues of strategic
importance (guidance and coaching of NCPCs/NCPPs, peer review of products and
services, innovation in services delivery, lesson learning, etc.).

The above-mentioned systemic internal constraints are not only faced by the UNIDO CP
Unit in its management and administration of the UNIDO-UNEP CP Programme.
However, they are important barriers for effective programme development and
management and to some extent they might explain why no strategy existed from the
beginning for how UNIDO and UNEP would deal with NCPCs after the direct support
has ended. Nevertheless, the lack of such a strategy is certainly one of the central
weaknesses of the Programme and needs to be addressed as soon as possible (see
recommendations in section 7.2).

Adaptive management
The UNIDO-UNEP CP Programme has not remained static over the years. While the
NCPC concept was not changed significantly and was kept as the core element of the
Programme, some new elements were introduced and efforts were made to overcome
identified barriers. Overall this shows a degree of adaptability in programme
management, based to a large extent on an active dialogue between programme
management, donors, NCPC Directors, local counterpart institutions and International
Reference Centres (IRCs). However, at the same time, it should be noted that not all of
the changes and modifications produced good results and in general a lack of strategic
and thematic leadership of the UNIDO-UNEP CP Programme limited the translation of
pilot project results and studies into a continuous development and consolidation of the
Programme. The most important issues in this context were:

x CP Finance: already relatively early in the programme it was noticed by the parties
involved that lack of finance (or of access to it) represented a major barrier for the
introduction of CP at the company level. UNEP with support from the Government of
Norway implemented a dedicated project with several studies and pilot projects with
NCPCs in five countries were carried out to develop resource materials (e.g. [36]).
UNIDO’s programme management also made an effort to strengthen in-house
cooperation with the investment promotion division.

The Swiss donor, partly in cooperation with the UNIDO-UNEP programme, launched
another successful initiative in this respect. It is the Green Credit Trust Fund of
SECO, which was launched in 2004 and is being implemented in several countries
and is producing valuable results (see country reports Peru and Colombia).

x EST Transfer: another problem area that was soon identified was the transfer of
Environmentally Sound Technologies (ESTs). Most of the CP options implemented
in companies fall into the category ‘low (or even no) investment’ or ‘good
housekeeping’. Programme management, in cooperation with donors, realised that
generally the effectiveness of the Programme in achieving further reaching process
changes and substitution of technologies was rather low. As an answer to this
problem two pilot projects were carried out in India and China, providing special
resources to the NCPCs in order to produce tangible and replicable results in the field
of EST transfer. Unfortunately these projects did not eventuate as expected. The
Indian project is still on track to achieve some EST transfer, but the Chinese project

28
 has been abandoned. Nevertheless, through implementing these projects, important
lessons were learned and barriers to EST transfer identified. Increasing effectiveness
in EST transfer (including adaptation and replication of ESTs), however, remains one
of the big challenges of the CP programme.

x Environmental Management Systems: at the outset of the CP programme in the early

1990’s Environmental Management Systems (EMS) were not yet widespread tools
(the British Standard BS 7750, which can be regarded the prototype of EMS, was
published in 1992, and its international companion ISO 14001 was published in
1996)). Thus it is not surprising that EMS did not figure prominently among the tools
to be employed by NCPCs in the original documents. However, the Programme,
through its direct relation to a set of International Reference Centres who are familiar
with state-of-the-art environmental practices in industrialised countries, introduced
EMS into the work of NCPCs quite successfully. Since then, EMS has become an
important service area and source of income in several NCPCs.

x New Services: in recent years the programme engaged more and more in the
introduction of new services in the NCPCs. This was partly motivated by the
aspiration of UNIDO and UNEP to fulfil its role as innovators and opinion leaders
within the global CP community. To some extent it followed the request from the
donors to introduce such services through the NCPCs. The relevance and
applicability of the different services from the perspective of the NCPCs/NCPPs will
be discussed in more detail in Section 3.3 (self-assessment results). From a
programme level perspective it is important to mention two issues related to the new
services. First, not all of these services fall into the CP concept (especially CSR and
consumption related services). Second, the two agencies involved, UNIDO and
UNEP, and the donors have not yet come to a common understanding what kind of
services should be offered by NCPCs. While UNIDO has established set of new
services under the heading of CP+, UNEP is aiming at the integration of sustainable
consumption related services to better match with its mandate for SCP. Overall it
would be necessary to strike a balance between perceived priorities of the countries
(as reported by the NCPCs/NCPPs) and the global priorities (reflected in the
mandates of the UN agencies).

The issue of new services has been discussed intensively with NCPCs, but at present no
clear strategy has been defined as to what should and what should not be part of the CP
Programme’s area of services. It appears that the approach so far was to ask ‘what could
the NCPCs do next to sustain themselves?” instead of asking ‘what needs to be done to
achieve widespread application of CP in a given national context?’ The latter approach
would imply more development of new and innovative methods and policies within the
Programme, based on a more regular and in-depth assessment of the demand and the
experiences in different countries, sectors, companies and activity areas.

Internal collaboration within UNIDO

It is noteworthy, that despite many attempts from project managers to enhance
cooperation with other relevant UNIDO technical branches (e.g. Investment Promotion,
ICS Trieste) no significant cooperation between the CP Unit and other areas of UNIDO
has been achieved so far. This is particularly surprising, given the CP-related activities of
UNIDO in the following environmental areas: Montreal Protocol, CDM and Climate
Change, Stockholm Convention. All these areas are closely related to the concept of CP.
The capacity built up through the UNIDO-UNEP CP Programme in many countries could

29
have been enhanced in these areas and NCPCs/NCPPs could have been invited more
frequently for the implementation of some of the before mentioned activities. To some
extent related to this situation and as shown by the results of the self evaluation survey
(see Section 3.3) the NCPCs/NCPPs report that their activity level and perceived
competence in the areas of multilateral environmental agreements (MEAs) is lowest in
the fields of Montreal Protocol and Stockholm Convention.

Several factors have contributed to this situation. First, UNIDO’s policy to deliver its
technical cooperation in so called ‘integrated programmes’ at the country level did, in
many cases, not succeed in establishing synergies and cooperation among modules at the
country level. Second, the cooperation at the programme level between different UNIDO
programmes has not been a high priority for management and no particular incentives
were created for such a co-operation. Third, the funding of UNIDO’s technical
cooperation in general is to a large extent based on project-by-project funding from
different bilateral donors. This situation does not only make programmatic approaches
difficult, it also represents a barrier to cooperation between programmes, as the alignment
of strategies and approaches to requirements at the project level reduces the flexibility of
UNIDO to optimise its programmes through synergies at the programme level.

Interagency collaboration
CP as a concept spans the mandates of several UN agencies. CP is at the core of the
mandates of UNIDO and UNEP and the leadership exercised by the two agencies reflects
well the combination of industrial development aspects with the need for environmental
sustainability. As mentioned before the LoA signed at the outset of the Programme was
the basis for the cooperation between UNIDO and UNEP. However, actual cooperation
between the two agencies was ad-hoc and depended to a large extent on the personal
relationship between key staff involved in both agencies. It was not based on a
mechanism with established procedures for joint programming, adaptive programme
management and decision-making on the basis of monitoring and evaluation at the
programme level. The absence of procedures and shared aims and objectives was further
complicated by differing mandates in both agencies (e.g. the question whether
consumption related issues should be dealt with by NCPCs).

In addition to the cooperation between UNEP and UNIDO, which is directly related to the
concept of CP, cooperation was established in a few countries with ILO for the provision
of environmental and CSR (corporate social responsibility) related services through
NCPCs within ILO’s ‘Factory Improvement and Decent Work Programmes’. The
cooperation is based on a Memorandum of Understanding (MoU) between the two
organisations and SECO. It was signed during the World Summit on Sustainable
Development in Johannesburg in 2002. The MoU foresees pilot cases in Swiss-financed
centres. Four such cases located in Latin American countries were evaluated on behalf of
SECO in 2005. The evaluation report concluded that the introduction of Corporate Social
Responsibility (CSR) on the basis of the ILO ‘Factory Improvement Programme (FIP)’
was largely successful and led to local adaptation of the FIP. No evidence has been
presented with regard to the collaboration strategy of the Programme in general and it is
not known to what extent cooperation was undertaken with some of the more important
multilateral and bilateral donor initiatives in the field of CP (e.g. GTZ, NORAD, IADB,
and ADB).

30
Donor Involvement and Fund Raising
One of the principal activities of UNIDO’s management of the UNIDO-UNEP CP
Programme was the raising of funds for the different NCPCs. The work effort related to
this activity was significant and resulted in a total funding volume of approximately USD
30 million (9) channelled through UNIDO to the different NCPCs. Additional funds were
channelled through UNEP to the NCPCs within multi-country projects. More than 60% of
this amount was provided by the two main donors: the Governments of Switzerland
(SECO) and Austria. Other donors of the programme were Brazil, Canada, Czech
Republic, European Union, Finland, Hungary, Italy, Japan, The Netherlands, Norway,
South Korea, Slovenia, Sweden and UK (www.unido.org/index.php?id=o5133).

The main donors of the UNIDO-UNEP programme liaised with programme management
with very different intensity. While SECO participated very actively, both at headquarters
and field levels in programme strategies, country selection and implementation (e.g.
through the development of indicators and evaluations of NCPCs commissioned by the
donor), the Austrian donor was mainly involved at the field level and left implementation
generally to UNIDO as the executing agency of the Programme. At the suggestion of
SECO, the programme management for example introduced business plans for the
NCPCs, and overall, these helped to improve viability of the NCPCs, as well as their
financial independence from the UNIDO-UNEP CP Programme.

The only example of funding that was not granted for a specific country is the
contribution from the Government of the Netherlands for the establishment of the first
five NCPCs. Given the long lasting partnership and the relation of mutual understanding
and trust between the two main donors of the Programme and UNIDO, it is surprising that
the positive experience of a more programmable funding from the beginning of the
Programme was not repeated at later stages of the Programme. This has limited the
potential to further develop the Programme and to move strategically to the next level.

Monitoring and Reporting

Monitoring and reporting has been done on a systematic basis mostly at the level of
individual NCPCs through reports prepared by the NCPCs/NCPPs to UNIDO. A set of
indicators was developed together with SECO and applied by the NCPCs/NCPPs since
1998/1999.

The quantity and relevance of these indicators as well as the rigour of application varies
widely and many NCPCs/NCPPs used erroneously expected benefits as a substitute for
effective benefits in their reporting (potential savings of CP options suggested to
enterprises were being reported as if they were savings actually made by enterprises).
Indicators on training were often purely quantitative and did not comprise indications on
the quality of training and achievements of trainees (test results, certification etc.). In
many cases no significant effort was made to ensure the accuracy and comparability of
data presented by NCPCs. As a result the information provided in annual reports of
NCPCs/NCPPs varies a great deal in quality and accuracy. This indicates that programme
management did not consider monitoring an important issue and did not pay enough
attention to results. This is definitely an important issue for the next stage of support and
should become a focus of attention.

9
Based on figures included in Table 2.1 , projects other than NCPCs not included

31
Starting in 2003 UNIDO has carried out an annual self-survey of the NCPCs. The survey
instrument was applied by an increasing number of NCPCs: in 2003 by 18 NCPCs, in
2004 by 26 NCPCs, in 2005 by 26 NCPCs and in 2006 by 23 NCPCs. The survey
represents a tracking tool of the current situation of UNIDO related NCPCs/NCPPs with
regard to the following information areas: industrial sectors covered by the NCPC;
staffing situation and expertise; type of services provided and demand for these services
(number of customers); CP relevant environmental legislation; degree of financial
independence and cost/income structure by service type; implementation level of
recommended CP options; and environmental benefits achieved (measured by a set of
quantity indicators).

That self-surveys have to be carried out to compile basic information on NCPCs

illustrates the fact that many NCPCs have no or very limited relation to the Programme
(hence they do not report to programme management) and those who have use different
formats and produce information of varying quality. If annual reporting of NCPCs/NCPPs
followed a common standard, there would be no need for generic surveys. It is
recognised, however, that those NCPCs that are no longer (partially) funded through the
UNIDO-UNEP CP Programme, cannot be expected to report to UNIDO and UNEP
unless there is a concrete benefit attached to it (such as membership in a network that
provides effective services to the NCPCs).

The application of the survey instrument is in principle a step towards more proactive
programme management as far as it does not duplicate existing information (contained in
annual or other reports). However, the quality of the returned survey questionnaires varies
highly and many lack the information that is more difficult to provide, i.e. the information
regarding outcomes and impact of the NCPCs’ interventions and services. Thus the
usefulness of the survey for a more results based management of the Programme is yet
limited.

Also UNEP has carried out surveys of NCPCs/NCPPs.[37]. They aimed at an assessment
of needs for support from UNEP and UNIDO rather than on results of CP interventions.
Such needs survey is certainly a good way of ensuring continued relevance of the support
provided by UNEP and UNIDO to the NCPCs.

2.3.2 National Centres

For analytical purposes the support provided by the programme to the NCPCs can be
divided into the following phases: pre-establishment phase, establishment phase, support
phase and post support phase. The cooperation activities of the UNIDO programme
concentrated mainly on the establishment and the support phases.

Prior to the establishment of the NCPC the cooperation of UNIDO was in the design of
the project document. UNIDO acted here as a partner for institutions in recipient
countries, offering its experience from other countries. This function of UNIDO is central
to the success and the sustainability of the NCPC. The cooperation during this phase was
not based on an analytical tool or method to determine the specific demand for CP
support in a country. It was based on the standard NCPC model and the personal and
professional judgement and experience of the UNIDO officers in charge, hence quality of
this support depended to a large extent on the availability of appropriate staff resources.

32
The cooperation during the establishment and the support phase was characterised by a
very deep and detailed involvement of UNIDO project managers in the daily operations
of the NCPCs. This included frequent participation of project managers in meetings of the
executive boards, revision and approval of business plans of the NCPCs, authorisation of
staff recruitment and participation in selection panels and close control of the NCPCs
budget (especially in the many cases where the bulk of the NCPC budget came from
UNIDO). With a growing number of NCPCs in the UNIDO-UNEP CP Programme this
close administrative oversight (“micro-management”) put considerable stress on the
limited resources of the programme management and diverted attention from strategic
and programme level issues to the project administration of individual NCPCs.

Some examples for activities during the post support phase exist, such as the two UNIDO
projects for transfer of EST carried out in cooperation with the Chinese and the Indian
NCPCs and the UNEP project on energy efficiency, implemented in 6 countries: China,
Vietnam, India, Hungary, the Czech Republic and Slovakia (EECPEMS: Energy
Efficiency through Cleaner Production and Environmental Management Systems). The
more recent strategy documents of UNIDO mention the importance of turning the NCPCs
into long-term partners for the implementation of UNIDO and other agencies’ projects.
So far this has not materialised to a significant extent.

2.3.3 Networking activities

There are several initiatives world-wide that try to offer some kind of networking to CP-
related institutions. These are brought together in the Regional Roundtables for
Sustainable Consumption and Production, which have been established in Europe (since
1994), Asia Pacific (since 1998), Africa (since 2001) and Latin America (ad hoc only).
Even though in many cases the NCPCs are active contributors to these roundtables, as are
UNIDO and UNEP, the Roundtables operate at arm’s length from the Programme and are
governed by independent boards at the regional levels.

At the global level, the UNIDO-UNEP network is certainly the most important
networking initiative. However, so far, the UNIDO-UNEP global network has been
exclusively based on those NCPCs/NCPPs that at some stage have received assistance
from UNIDO-UNEP.

The Programme has not addressed the important issue of institutional status of ‘UNIDO-
UNEP NCPCs’. The question for how long after the UNIDO-UNEP assistance a NCPC
can or should use the respective UN logos has not been answered yet. The introduction of
quality standards to ensure that only compliant NCPCs are eligible participants for the
network has been discussed but no concrete steps have been taken for the implementation
of such standards. Furthermore the NCPCs have been frequently referred to as ‘UNIDO-
UNEP Centres’ (10), which suggests at least a certain institutional relationship between
the Centres and the UN agencies. Such a relationship does not exist in many of these
Centres, in particular those who do not receive any further funding from UNIDO or

10
For example: “Operationalising UNIDO Corporate Strategy”, p. 66 “the Organization will continue to develop the
technical cooperation services offered through its worldwide network of National Cleaner Production Centres (NCPCs)
and National Cleaner Production Programmes (NCPPs).”; or page 83: “The cleaner and sustainable production (CP)
strategy of UNIDO aims at utilizing the National Cleaner Production Centres (NCPCs) to implement the following two
specific sets of interventions:…..”

33
UNEP. This situation is of particular concern and requires urgent action from UNIDO-
UNEP management.

Some other important unresolved issues related to the global networking are:

x The issue of exclusiveness: who is or could be a member of the network is not

entirely clear at the moment. For example, are members of a regional network, like
the network in Latin America, who have not received funding through the UNIDO-
UNEP CP Programme automatically members of the global network? Should there be
only one member per country (normally the NCPC) or could several institutions
(including regional, local and or sectoral CP Centres) participate?

x Complementary operation of the global and regional networking initiatives: currently

the programme supports both types of initiatives without a clear strategy and
definition of roles, thus creating a risk of duplication and reduced efficiency.

x Types of members: should only institutions be members of the network or could CP

consultants

Box 2.1: Regional networking

Networking and co-operation among NCPCs has happened ad-hoc and as a result of specific interventions in-
and outside the UNIDO/UNEP programme. Several examples exist where the programme facilitated the co-
operation between individual NCPCs on a project basis but also in some cases mature NCPCs helped new
ones to build up its capacity (e.g. Viet Nam NCPC in the case of Lao PDR and Cambodia). Examples for
networking outside the UNIDO/UNEP programme are GTZ funded networks in Latin America and Africa
(e.g. the ‘Andres Bello Network for CP in Latin America’) or the regional CP roundtables in Africa, Asia,
Europe and Latin America.

Within the UNIDO/UNEP programme, the CP LatinNet is the most important regional networking initiative
so far. UNIDO, with cooperation of UNEP, is executing a project to set up and strengthen the regional
network, which aims at ‘the establishment of an efficient Latin American & Caribbean CP Network that
promotes the increased application of a holistic and sector Cleaner Production approach and enhances
Environmentally Sound Technology transfer’. Funding for the initial period is provided by the Austrian and
Swiss Governments. The long-term plan foresees the network to become self-administered and sustainable on
the basis of membership-fees. The incentive for the individual Centres to contribute to the CP LatinNet is
based on the expected benefits to be derived from a set of activities:
x Regional projects: NCPCs cooperate in the design and joint implementation of regional projects;
x A Mechanism of regional experts exchange;
x Joint promotion of the Regional Programme to obtain additional members and interest from donors;
x A Knowledge Management System; and
x Training and CP awards
The evaluation team found that the progress of this initiative is encouraging. Most of the 14 members have
paid their fees into a trust fund and an information management platform has been established currently
including more than 500 technical documents in the database, accessible for close to 400 registered users.
However, most of the stakeholders (especially member NCPCs) have expressed concerns about the
effectiveness, efficiency and sustainability of the network and not much progress has been made yet in the
development and implementation of regional projects. Before replicating the experience of LatinNet in other
regions, it would seem advisable to carry out a mid-term evaluation of the LatinNet initiative, with special
reference to the overall role of regional networking within the UNIDO/UNEP programme and vis-à-vis other
networking initiatives at the global and regional levels.

and professionals join in? Should institutions from developing countries have a
different status from such in industrialized countries and should the latter be
members at all?

34
As mentioned in section 2.2.2., the objectives and the strategy for the global networking
activities of UNIDO have not been made entirely explicit and specific funding was
available only on an ad-hoc basis, primarily for the organisation of the ‘annual Directors’
meetings’ (see below). These meetings of the NCPC directors and a number of CP experts
were the most important networking activities of the Programme. They were designed to
facilitate the sharing of information, the dissemination of best practices among
NCPCs/NCPPs and the participation of NCPCs/NCPPs in the strategy discussions at the
programme level.

UNIDO has also supported the establishment of a regional network of NCPCs in Latin
America (see box 2.1). The regional network has a number of interesting features that go
beyond the services currently offered by the global network. More importantly, the
regional network is open to such institutions that have never been part of the UNIDO-
UNEP global network and who have not received any assistance through these agencies.

2.3.4 Technical Assistance

Throughout the Programme specialised firms or institutions with experience in CP

supported the institutional capacity building of the NCPCs. These acted as International
Reference Centres (IRCs). For the ‘multilateral’ NCPCs (those that come under UNIDO-
UNEP programme management) the two functions of administrative management and
technical assistance were separated, i.e. UNIDO was in charge of the administrative and
institutional management (e.g. budget and disbursement to the Centre, contracts of NCPC
staff, monitoring of NCPC performance, participation in the executive board of the
NCPC) and the IRCs provided technical inputs (e.g. training, advice for in-plant
assessments). For the ‘bilateral’ NCPCs (those without UNIDO-UNEP involvement)
both of the above mentioned two functions were exercised by the IRC. It is however
worth mentioning that typically the bilateral centres have been funded at levels 3 to 4
times higher than the NCPCs established under the UNIDO-UNEP CP Programme.

Both arrangements have their pros and cons. While the multilateral approach is less
efficient and leads to longer administrative processes, the bilateral approach puts two
ideally separated functions (administration and technical advice) under the responsibility
of the same institution, thereby limiting the potential for a beneficiary country driven
delivery of consultancy services. There is no clear evidence that one of the two
approaches is more effective with regard to the creation of national capacity. The cases
analysed show that NCPCs can be established by bilateral agencies as well as by
multilateral one with similar results (see analysis of independent country evaluations and
overall programme assessment). In both cases, there is a tendency to engage too much
and for too long a period in the administrative ‘micro-management’ of the NCPCs,
including the recruitment of national staff and the management of the operational budgets
of the centres.

The multilateral approach harbours a greater potential for creating a growing pool of
IRCs with broad sectoral and country experience. This was actually envisaged to be
turned into a main value added of the Programme over time [27]. Whether or not such an
effect has materialised is not fully clear (no specific reporting available on this).
However, anecdotal evidence exists that IRCs that have delivered successful services to
some centres, are later on used by other NCPCs to deliver the same services. A case in
point is the successful ECO Profit model promoted by Stenum. On the other hand, the
multilateral approach adds complexity to the overall management of technical assistance,

35
sometimes obstructing a more efficient direct relationship between reference centres and
NCPCs.

2.3.5 Publications and Information Management

The CP programme has produced a considerable number of manuals, training materials,

sectoral CP guides and issue papers on specific CP issues. UNEP and UNIDO maintain
specific CP websites offering most of these documents to the public (see also section 2.4.
and the self evaluation of publications by the NCPCs/NCPPs in section 3.3)).

The information produced and provided by the Programme has not yet been assembled
into an information and/or knowledge management system as in the case of the CP
LatinNet. Prior to the launch of the UNIDO-UNEP CP Programme, UNEP operated in
the 1990’s the “International Cleaner Production Information Clearinghouse” (ICPIC) –
an compilation of case studies, technical manuals and fact sheets on CP, initially provided
by the United States Environmental Protection Agency, but complemented with the
results from UNEP CP working groups. It was planned that the NCPCs would contribute
their results and experiences into ICPIC, but this did not materialise. With the increased
availability of the Internet, UNEP has discontinued ICPIC.

2.4 Programme Results

As per the re-constructed logical framework for the UNIDO-UNEP CP Programme

results include the different dimensions, namely outputs, outcomes and impact (see also
Figure 2.1). This section covers outputs at programme level. Outcomes and impacts are
achieved principally at the national level in the host countries and are therefore analysed
mainly on the basis of the independent country evaluations undertaken by the evaluation
team (see Chapters 4 and 6).

2.4.1 NCPCs

The implementation of the CP programme foresaw the establishment of 20 NCPCs in a

five-year period from 1994 to 1999 in two phases. In Phase I the first NCPCs in China,
India, Mexico, Tanzania and Zimbabwe (all funded by The Netherlands) were
established. This was then followed by the NCPCs in Slovak Republic and Czech
Republic (funded by Austria), and Brazil (self financed by Brazil). These NCPCs have
been referred to frequently as the ‘first generation NCPCs’ (or ‘old’).

After this first set of NCPCs was established, the goal to create 20 NCPCs was achieved
according to plans and later on further 14 NCPCs/NCPPs were established by the
UNIDO-UNEP CP Programme. In 2007 another four (Bulgaria, Romania, Serbia and
Montenegro) are either in the phase of NCPP or at the initial phase of NCPC
establishment. Table 2.1 shows the 37 countries covered by this evaluation with the
respective funding amounts. Another three countries are on the NCPC list (Armenia,
Panama and Paraguay) but no budgetary or management information was available to the
UNIDO CP Unit. The list includes four NCPCs that have not received any funding
through the UNIDO-UNEP CP Programme but were to some degree connected to it either
through original support in the design of a NCPC support project that then led to funding
by a different source or through the bilateral funding through the SECO Programme on

36
Sustainable Enterprise Development Centres which maintained close cooperation with the
UNIDO-UNEP CP Programme.

Table 2.1: Donor funding for NCPCs

UNIDO Start (expected) End of Donor Total Amount Amount per
NCPCs Operation UNIDO/Donor received (USD) Year (12)
support (11) (USD)
Armenia 2005 2007 Austria 221,240 110,620
(NCPP)
Bolivia^ 1995 2007 (ongoing) Switzerland, - -
USA, Denmark
Brazil 1995 1998 Brazil 330,000 110,000
Cambodia 2004 2007 (extension Switzerland 802,000 267,000
planned)
China 1995 1998 The 310,000 103,000
Netherlands
Colombia^ 1998 2006 Switzerland 2,800,000 400,000 13
Costa Rica 1998 2006 Switzerland 1,854,000 206,000
Croatia 1997 1999 Czech 175,000 58,100
Cuba 2001 2007 Austria 596,000 (4 155,000
years)
490,000 (3
years)
Czech 1994 1999 Austria 603,000 120,600
Republic
Egypt 2004 2010 Austria, 600,000 150,000
Switzerland
El Salvador 1999 2006 Switzerland 1,860,000 232,500
Ethiopia 2000 2008 Italy 900,000 (incl. 100,000
extension to
2008)
Guatemala 1999 2006 Switzerland 1,588,000 198,500
Honduras^ 2000 2005 Canada *
Hungary 1997 2001 Austria 404,000 101,000
India 1995 1998 The 310,00014 103,000
Netherlands
Kenya 2000 2004 UNDP 637,200 127,440
Laos 2004 2007 (extension Switzerland 769,000 256,000
planned)
Lebanon 2002 2008 EU/Austria 310,000 52,000
Macedonia 2001 2007 Czech 300,000 50,000
Republic,
Austria
Mexico 1995 1998 The 310,000 103,000
Netherlands
Morocco 2000 2007 Switzerland 1,580,000 226,000
Mozambique 2000 2007 Italy 678,000 84,750
Nicaragua 1997 2007 Austria 1,561,00015 156,100
Peru^ 2002 2007 (ongoing) Switzerland, 1,800,00016 360,000
USA
Republic of 2001 2005 Republic of 593,000 118,600
Korea Korea

11
Includes technical assistance provided by international experts or International Reference Centres, rounded figures,
source: UNIDO infobase as of October 2007
12
Total amount received divided by duration of funding support period
13
Approximation from budget data of annual reports
14
Does not include specific project on cleaner technology promotion
15
Includes specific project on Sustainable Industrial Resource Management (SIRM)
16
Estimated amount, no exact figures for contributions from both donors available

37
Russia 2001 2007 United 1,068,000 178,000
Kingdom,
Austria
Slovakia 1995 2001 Austria 513,500 86,000
South Africa 2002 2007 Switzerland, 1,619,000 324,000
Austria
Sri Lanka 2001 2007 Norway 1,030,000 172,000
Tanzania 1995 1998 The 310,000 103,000
Netherlands
Tunisia 1996 1998 Norway 66,500 33,200
Uganda 2001 2007 Austria, 1,586,000 264,000
Norway
Uzbekistan 2005 2007 Austria 102,000 34,000
Vietnam 1998 2007 Switzerland 3,985,000 443,000
Zimbabwe 1995 1998 The 310,000 103,000
Netherlands
^ these NCPCs have not had funding support through the UNIDO-UNEP CP Programme

With funds mobilisation being one of the most important outputs of programme
management, it is obvious that a lot has been achieved in this respect. At the same time it
should be made clear that more of the limited time and resources of programme
management could have been devoted to more substantive issues if funding would have
been available at the programme level. The annual support provided to NCPCs ranges
from USD 33,000 (Tunisia) to USD 443,000 (Vietnam), indicating a wide range of
funding volumes employed to support NCPCs. Also the duration of funding support
varies widely (between 3 and 9 years).

The volume and duration of support can be compared with ‘ex-ante’ criteria, i.e. such that
are commonly applied in the planning stage of an NCPC (country size, level of industrial
development, importance of industrial pollution, etc.; see for a more detailed discussion
Chapter 5, portfolio analysis) and ‘ex-post’ criteria, i.e. primarily the level of success in
terms of sustainability and effectiveness. With regard to the ex-ante comparison it can be
observed that there is no correlation between the volume of funding and the size of
environmental and economic challenges to be addressed by the NCPC. Some small
countries with relatively limited industrial pollution, like the ones in Central America,
received relatively high and long support, while some big countries with significantly
higher environmental pollution problems (e.g. China, India, and Mexico) received
relatively low and short support.

With regard to the ex-post analysis Table 2.2 provides an overview of the NCPCs
reviewed by this evaluation, including past and present linkages to the UNIDO-UNEP CP
programme and current status of the NCPC as a leading agency in its country (17). Also
here no easy lesson can be learned. There is no correlation between the fact that a
NCPC/NCPP has positioned itself as lead agency and the volume or duration of financial
support. However, the fact that funding support through the Programme is still ongoing
shows a clear relation to the strength of the current linkage between the UNIDO-UNEP
CP programme and the respective NCPCs. All eight NCPCs or NCPPs listed as
maintaining a strong relationship with the programme are currently receiving funds
through the Programme. While this might seem obvious, it clearly indicates that the
Programme so far has not been able to establish a substantive relationship to NCPCs
beyond the funding period.

17
The ratings are based no the judgment of the evaluators. In some cases no such judgment could be formed due to lack of
information.

38
Apart from the support to the establishment of NCPCs by UNIDO, UNEP also
implemented a number of projects in cooperation with NCPCs, primarily to test new and
innovative approaches to enhance the application of CP. The most important of these
projects are:

x Cleaner Production Financing, In 1999 UNEP started a four-year project aiming at

increasing investments in cleaner production in developing countries. The project,
focused on five demonstration countries: Guatemala, Nicaragua, Tanzania, Vietnam
and Zimbabwe and was conducted under a trust fund created by the Norwegian
Government.

x Cleaner Production/EE projects: ‘Promoting Industrial Energy Efficiency through a

Cleaner Production/Environmental Management System Framework’ (EECPEMS).
The pilot projects were carried out in six countries: China, Vietnam, India, Hungary,
The Czech Republic and Slovakia.

x The follow up project ‘Greenhouse Gas Emission Reduction from Industry in Asia
and the Pacific’ (GERIAP) was established to develop and apply a CP-EE
methodology in four energy-intensive sectors in the Asia Pacific region and was
supported by the Government of Sweden.

x Project on CP and Multilateral Environmental Agreements (ACME) building capacity

in India and Ukraine to use CP to support implementation of Multilateral
Environmental Agreements (MEA).

39
Table 2.2: Results of NCPC/NCPP establishment
a leading agency?
Techni- Institu-
Country past linkage to UNIDO-UNEP current linkage to UNIDO-UNEP cally tionally
Bolivia none marginal
Brazil Strong marginal
Cambodia Strong strong
China Strong none yes yes
Colombia none marginal yes no
Costa Rica Strong marginal yes yes
Croatia Strong marginal yes
Cuba Strong medium yes yes
Czech
Republic Strong marginal yes
Ecuador none none no no
Egypt Strong strong
El Salvador Strong medium yes yes
Ethiopia
Guatemala Strong medium yes yes
Honduras none marginal no no
Hungary Strong none no no
India Strong none yes yes
Kenya Marginal none yes
Laos Strong strong
Lebanon Strong marginal
Macedonia Strong strong
Mexico Strong marginal yes no
Morocco Strong yes yes
Mozambique Strong medium yes no
Nicaragua Strong medium yes yes
Peru none marginal no no
Rep of Korea
Russia (St.
Petersburg) Strong medium yes yes
Slovak
Republic Strong marginal yes
South Africa Strong marginal no no
Sri Lanka Strong strong no yes
Tanzania Strong none
Tunisia Marginal none yes yes
Uganda
Uzbekistan Strong strong
Vietnam Strong strong yes no
Zimbabwe Strong none
7 2 0 28 9 13 6 8 64% 58%

Source: Assessment by evaluation team

x Norwegian Project to support establishment of an African Roundtable on Cleaner

Production.

x CP in the African Brewery Sector (ABREW, a first stage demonstration project for a
larger scale second phase). The project includes demonstration CP assessments in two

40
 breweries in Uganda and a Pan-African review of the potential for CP in the African
brewery sector.

x Finnish Task Force on Sustainable Buildings and Construction and UNEP

Sustainable Buildings and Construction Initiative (SBCI): Finland is hosting the
Marrakech Task Force on Sustainable Buildings and Construction, in which UNEP’s
Sustainable Buildings and Construction Initiative (SBCI) is a close partner and has
provided substantial support. Involves a compilation of a list of joint policy
recommendations for the CSD (Commission for Sustainable Development) in May
2007 and the publication of best policy practices. UNEP Sustainable Building and
Construction Initiative (SBCI) is a close partner of the Finnish TF and they have
jointly published a baseline report entitled Buildings and Climate Change: Status,
Challenges and Opportunities in 2007.

x UNEP-InWEnt projects on capacity building in Cleaner Production Centres.

x Application of Environmental Technology Assessment (EnTA). NCPCs were trained

in a methodology to assess environmental technologies.

2.4.2 Networking

As mentioned in paragraph 2.3.3 the most important networking activities were the
international meetings for the directors of the NCPCs/NCPPs organised by UNIDO and
UNEP. Table 2.3 lists the time and location of the nine annual meetings that have been
held since establishment of the Programme in 1994 (13 years).

Table 2.3: NCPC ‘Annual’ Meetings

NCPC Annual Meetings have been held as follows:
Place Host Country Date
Vienna Austria 13-15 December 1995
Nyanga Zimbabwe 25-30 November 1996
Bangkok Thailand 6-9 November 1997
Prague Czech Republic 7-12 March 1999
Berne Switzerland  7-12 May 2000
Seoul Republic of Korea 5-9 November 2001
Mayrhofen Austria 7-9 May 2003
Interlaken Switzerland 7-12 June 2004
Semmering Austria 24-26 September 2007
Source: UNIDO website

In addition to these annual meetings UNEP carried out the following networking
activities in the period between 1992 and 2005:
x 8 International high level Seminars on (Sustainable) Cleaner Production;
x Support to S(CP) regional roundtables: twenty interventions;
x International Declaration on Cleaner Production (incl. signing ceremonies;
declaration brochure and poster); and
x CP website (on-going).

41
It is very difficult to assess the effectiveness of such meetings, since the benefits of social
interaction between CP experts can hardly be quantified. However, from interviews with
NCPC Directors it can be concluded that the annual meetings are a valuable source of
information and experience exchange between professionals.

The UNIDO-UNEP CP Programme has made an important contribution to the

development of an international CP community by bringing together experts on CP from
all over the world and by facilitating the experience exchange between these experts. In
this context it should be noted that the main part of the technical assistance of the
Programme was provided by a select number of International Reference Centres, i.e.
qualified institutions with experience in different fields of CP (see Table 2.4). The
relation between the CP Programme and some of these institutions was maintained
throughout the programme. While no in-depth analysis has been undertaken of the
interactions between these centres and the programme, it can be said that the approach to
establish long-term relationships with internationally renowned institutions is mutually
beneficial (to some extent the Programme might also have helped these institutions to
position itself in the international community of CP) and represents a best practice. It is
considered more effective and sustainable than relying on a network of individual
consultants as is the case in many other UNIDO programmes.

The use of the select group of IRCs appears to have been beneficial for fostering
coherence in programme implementation among recipient countries, and the use of more
experienced NCPCs as IRCs for newly established NCPCs/NCPPs is being applauded.
With the maturing of the Programme, more attention is needed to expose NCPCs/NCPPs
to different methods and practices for CP service delivery, and thereby enable
NCPCs/NCPCs to develop methods and practices that are most suited to the local
circumstances in their home countries (see also portfolio analysis in Chapter 5).

Table 2.4: International Reference Centres utilised by the UNIDO-UNEP CP Programme

International Reference Centres Country Period of
Service
Delivery (*)
IVAM Environmental Research, University of Amsterdam The 1995-1998
Netherlands
Erasmus Centre for Environmental Science, Erasmus University The 1995-1998
Netherlands
Danish Technological Institute Denmark 1995-1996
Danish Technological University Denmark 1995-1998
University of Massachusetts at Lowell USA 1995-1998
World Cleaner Production Society Norway 1995-1997
STENUM Austria 1995-ongoing
Fach Hochschule Nordwest Schweiz (University of Applied Life Switzerland 1998-ongoing
Sciences, Northwest Switzerland) (FHNW, formerly FHBB)
EMPA Switzerland 1998-ongoing
Bob Partners Switzerland Ongoing
Urbaplan Switzerland Ongoing
Slovak Cleaner Production Centre Slovakia Ongoing
Czech Cleaner Production Centre Czech Ongoing
Republic
(*) This refers to the period of active engagement as an International Reference Centre for any of the NCPCs. This excludes
some minor project-related consultancies through UNEP projects supporting NCPCs and/or collaborative projects between
IRC and selected NCPCs outside of the UNIDO-UNEP CP Programme.

42
2.4.3 Resource Materials

UNIDO and UNEP have produced a large number of resource materials for the
NCPCs/NCPPs (training tools, guidelines, sectoral CP guides, etc.). These are typically
also available to CP service providers outside the UNIDO-UNEP CP Programme. The
usefulness of the most important of these resource documents has been analysed based on
a survey among NCPCs (see section 3.3 for a detailed analysis).

In accordance with the originally envisaged division of labour between UNIDO and
UNEP, the latter has been more active in producing such materials and in some cases in
providing the corresponding training to NCPCs. Some examples are:

x Support to the development of the D4S (Design for Sustainability) Manual and
UNEP’s D4S activities. Including publication of ‘D4S A practical approach for
emerging economies’ [38] .

x How to use Environmental Management Tools (called Environmental Management

Navigator). With Wuppertal Institute, NCPCs were trained on this web-based tool
that explains a number of environmental management tools and how they can be best
applied.

x Facilitating implementation of Multilateral Environmental Agreements through

Cleaner Production, Integrating Cleaner Production and Sustainable Consumption.
Both modules were delivered to NCPCs.

x Cleaner Production and Environmental Management in Industrial Estates (follow up

project with Slovak NCPC). The module was developed and initially given in the
Philippines and piloting is being carried out in Slovakia, one of the NCPCs that
attended the first training.

x Building upon UNEP's projects in the area, a training package was prepared
(Energising CP) [39].

Both organisations have organised their information on CP on their respective websites.

No joint website and no central information management system exist for the UNIDO-
UNEP CP Programme.

2.5 Key Findings

2.5.1 Quality of Design

The concept of CP is well reflected in the design of the Programme and originally the
Programme was a coherent approach to building CP into an international cooperation
initiative. Over time, the consistency and clarity of the Programme has diminished to
some extent, given the frequent attempts to re-design and re-shape the Programme,
without a clear strategy and logical framework. Simultaneously, insufficient provisions
were made to ensure ongoing input from both UN agencies over time in particular on
strategic matters.

43
The NCPC model can be described as largely successful and demand oriented, given its
replication at a large scale and the continued demand for the set up of new NCPCs.

Not all of the interventions of the Programme will contribute to poverty alleviation,
productivity gains and environmental impact reduction to the same extent. Thus, when a
component or project is designed it should be clearly stated which of the Programme’s
goals is being primarily aimed at.

Already the NCPC Programme evaluation carried out in 1996 recommended the
establishment of a ‘firm programme concept’ and the ‘establishment of a dialogue
between UNIDO-UNEP and the partners’ [22]. The evaluation team concludes that the
good potential of the Programme for increased effectiveness and relevance can be
exploited fully only if a solid programming exercise is carried out.

A strategy to deal with NCPCs that do no longer receive funds through the Programme
does not exist and this presents a major weakness, which should be addressed by the
UNIDO-UNEP CP Programme in the immediate future.

2.5.2 Quality of Implementation

The UNIDO-UNEP CP Programme started in the early 90’s with a strong programmatic
approach, including a clear strategy and the target to set up of 20 NCPCs in the medium
term. Over the years, this programmatic approach has weakened considerably and was
replaced by a focus on the implementation of individual CP projects (mainly set up of
NCPCs) with little steering and monitoring at the programme level. This approach has led
to the establishment of 34 NCPCs and NCPPs worldwide and a continued demand for the
establishment of new centres. On the other hand, the reduced importance given to
programme aspects (including systematic programme-level planning, monitoring and
evaluation) has limited the potential of the UNIDO-UNEP CP Programme to build on
past experience for improved quality and effectiveness of CP interventions and to
exercise thematic leadership within the Programme as well as in the broader international
community. Also the relatively limited internal (within UNIDO) and external (inter-
agency) cooperation in the UNIDO-UNEP CP Programme represented a barrier for wider
impact at the programme level.

The main reasons for these shortcomings are UNIDO-internal systemic constraints and a
general lack of programmatic funding. The institutional status and the objectives of
networking activities also need urgent clarification.

The provision of technical assistance through the UNIDO-UNEP CP Programme has

been largely effective and of good quality. Efficiency however needs to be improved by
reducing the degree of micro-management (in particular on administrative matters) and
centralised agency execution and by establishing a more direct relationship between
NCPCs as contractors and international reference centres as technical advisors.

The UNIDO-UNEP CP Programme has produced a large number of outputs and valuable
outcomes. A commendable effort has been made to support the establishment of NCPCs
in more than 30 countries and the sustainability of these efforts is considered good. The
main contribution of the programme to the institution building at country level has been
in the planning and funds-mobilisation as well as in the organisation of technical
assistance to the NCPCs.

44
So far the programme has been less effective in the field of networking and up-stream
services. Efficiency has been relatively low, given the systemic constraints inherent in the
current modalities of technical cooperation through multilateral agencies.

45
3
Self Evaluation
_______________________________________

3.1 Introduction

The second ‘pillar’ of the independent evaluation reported here was a self-evaluation by
the centres. The primary aim of the self-evaluation was to obtain comparable baseline
information on the operation, management and activities of all NCPCs/NCPPs directly
from the Directors who run these on a daily basis. The secondary aim was to assist with
the selection of countries to be visited by a member of the international evaluation team
to undertake an independent country evaluation (as covered in Chapter 4 of this
evaluation report).

The self-evaluation was based on two independent surveys conducted by email among the
nominated Directors of 38 NCPCs/NCPPs covered by this programme evaluation.

ƒ Survey 1: a broad based survey into the current status of the NCPC/NCPP, covering
management information, activity information, results and assessment.

ƒ Survey 2: a specialist survey into emerging topics and tools in the UNIDO-UNEP CP
Programme. It was undertaken in response to suggestions at the first meeting of the
Steering Committee to assess in greater detail the level of interest, expertise and
experience of the NCPCs/NCPPs, in regard to such new service areas, Multilateral
Environmental Agreements (MEAs) and resource materials (publications and training
materials).

The first survey was issued immediately after the launch of the evaluation study (on 20
April 2007), and after repeated follow up, a total of 36 responses had been received by 7
October 2007. The two missing responses are Costa Rica (but Costa Rica was included in
the list of countries visited for an independent country evaluation) and Ethiopia (no
information obtained at all). For each respondent a country profile was compiled, and
these are available on request from the UNIDO Evaluation Group.

The second survey email was distributed on 10 July 2007, and after repeated follow up, a
total of 23 responses had been received by 20 September 2007. The responding countries
are listed in Table 3.1.

Table 3.1: Survey responses

Region [total Respondents
number of First Survey Second Survey
NCPC/NCPPs] [total responses] [total responses]
ƒ Africa [10] Egypt, Morocco, Mozambique, South Egypt, Kenya, Morocco,
Africa, Tanzania, Tunisia, Kenya, Mozambique, Tanzania and

47
 Uganda and Zimbabwe [9] Zimbabwe [6]
ƒ Asia [9] Cambodia, China, India, Laos, Lebanon, Cambodia, China, India,
Republic of Korea, Sri Lanka, Laos, Lebanon, Republic of
Uzbekistan and Vietnam [9] Korea, Uzbekistan and
Vietnam [8]
ƒ Central America Cuba, El Salvador, Guatemala, El Salvador, Guatemala
[8] Honduras, Mexico, Nicaragua and Mexico and Nicaragua [4]
Paraguay [7]
ƒ Central Eastern Armenia, Croatia, Czech Republic, Croatia, Czech Republic and
Europe [7] Hungary, Russia North West Region (St Slovakia [3]
Petersburg), Russia (Oil & Gas Centre,
Moscow) and Slovakia [7]
ƒ South America Bolivia, Brazil, Columbia and Peru [4] Bolivia and Colombia [2]
[4]
Total: 38 Total: 36 responses [95%] Total: 23 responses [61%]

The lower, but still very acceptable, response level for the second survey most likely
reflects that fewer NCPCs/NCPPs have experience on the expanded set of topics covered
in the second survey, while also a degree of survey-fatigue among the NCPCs/NCPPs
may have been at play. The responding countries appear an illustrative sample of
NCPCs/NCPPs in regard to their geographic, location, size and age, but no further
analysis was performed to confirm that the respondents were a representative sample of
all NCPCs/NCPPs in the UNIDO-UNEP CP Programme.

The findings from both surveys are summarised and reviewed here in an integrated
manner. First, section 3.2 covers management, governance and institutional issues.
Section 3.3 then covers the activities and services of the NCPCs/NCPPs, and section 3.4
covers the self-evaluation from the Directors on the competencies of their centres and
against the evaluation criteria set for this programme evaluation.

3.2 Management Information

Table 3.2 contains the data for the history of the NCPCs/NCPPs on the basis of their
reported establishment date. There are two peak periods in which most were established,
respectively a first wave in 1993-1995 (9 Centres) and a second wave in 1999-2001 (14
Centres). A relatively large share of the current NCPCs/NCPPs should be regarded as
mature; 28 (78%) were established prior to 2002 and thus have each an operational
history of at least 5 years.

Table 3.2: Reported establishment date for the NCPCs/NCPPs (36 responses)
Year
‘93 ‘94 ‘95 ‘96 ‘97 ‘98 ‘99 ‘00 ‘01 ‘02 ‘03 ‘04’ ‘05 ‘06 ‘07
New 2 1 6 1 1 3 4 6 4 3 1 2 1 1 0
Centres
Established
Total 2 3 9 10 11 14 18 24 28 31 32 34 35 36 36
Centres

The current institutional set up of these NCPCs/NCPPs is summarised in Table 3.3. The
majority of the Centres (61%) operates with limited independence, either as subsidiary of
the host organisation (44%) or otherwise semi-autonomously (17%). Only 31% of
NCPCs/NCPPs operate fully independently. In their operation, many therefore adopt the

48
legal status of their host. A large share of Centres operates with legal status of a public
entity (36%) or other NGO (typically a business association, respectively 17%). The host
institutions are quite diverse, but public sector entities prevail with 14% hosted in a
University, 19% in a Ministry/Department and 25% in other public entities. The large
shares of the other categories for legal status (30%) and host institution (28%) are
reflective of the fact that the institutional status of these centres is not resolved (for
example operating as a joint project of different public and/or private sector entities), does
not follow any of the standard categories used for the survey and/or that categories are
understood differently within the respective national legal systems. Overall however,
greater clarity on institutional set up would add to the achieving stability for the
NCPC/NCPP and ultimately the sustainability of the CP programme in the

Table 3.3: Institutional information (36 responses)

Degree of Independence Legal Status Host Institution
Fully 11 31% Association 3 8% Industry 4 11%
independent Association/
Chamber
Semi 6 17% Other Non Governmental 6 17% University 5 14%
autonomous Organisation
Subsidiary 16 44% Registered Private 1 3% Ministry/ 7 19%
of existing Company Department
organisation
Unknown 3 8% Public Entity 13 36% Other 9 25%
Public
Entity
Other 11 30% Other 10 28%
Unknown 2 6% Unknown 1 3%
Total 38 100% Total 36 100% Total 36 100%
respective countries.

Most of the NCPCs/NCPPs reported to have some kind of a board to guide their activities
(32, or 89%). These include broadly constituted advisory boards (28%), smaller
management or governing boards (44%) or steering committees (typically tri-partite with
only host and donor governments represented, and UNIDO and the NCPC) (17%).

Table 3.4 provides the summary data provided by the Directors on institutional funding
for their NCPCs/NCPPs received through the UNIDO-UNEP CP Programme. These
could within the context of this programme evaluation not be reconciled with
management records of the UNIDO CP Unit. Six respondents (17%) reported to have
never received institutional funding through the UNIDO-UNEP CP Programme. Those
that received institutional funding typically did so for 3 to 4 years (respectively 19% and
17% of respondents). However, some NCPCs have received institutional funding support
for much longer (17% received institutional funding support for 7 or more years). 11 of
the 30 countries that have been institutionally funded through the UNIDO-UNEP CP
Programme still received support in 2007. 19 have continued to operate without
institutional funding. About one third of these (37%, 7 countries) are in their first year of
operation without institutional funding. However a considerable number has continued to
operate without institutional funding for considerable time, for example 10 (33% of the
Centres once funded) now operate for five or more years without institutional funding.

49
Table 3.4: Centres by institutional support cycle (36 responses)
Number of NCPCs/NCPPs Total Number of Years
0 1 2 3 4 5 6 7 8 9 10 11
ƒ Length of institutional funding 36 6 0 3 7 6 4 4 2 1 1 1 1
period
ƒ Length of operation after 30 11 7 1 1 0 1 1 1 2 4 1 0
institutional funding

There is a distinct underlying pattern in the funding. The first batch of NCPCs was funded
by the Governments of The Netherlands and Austria (Brazil, Czech Republic, China,
India, Mexico, Slovakia, Tanzania and Zimbabwe). With the exception of India and
Tanzania, all of these NCPCs received relatively low institutional funding through the
UNIDO-UNEP CP Programme, and support was only given for an initial period of 3
years. Most of the centres established thereafter have been able to secure higher funding
levels in the first period and a second or even third institutional funding phase, implying
much longer and higher financial support. Austria and Switzerland have been and are the
main donors, as they contribute to the funding of respectively 12 and 11 Centres. There
are also a number of smaller donors that contribute funding only to one or two Centres,
e.g. Italy, Canada, Hungary, Czech Republic, European Union, United Kingdom and
Norway.

The Directors also reported on the total institutional funding they received. Responses
were obtained from 22 countries showing a range of USD 60,000 to USD 4.2 Million,
with an average of USD 863,000. These responses are not internally consistent and there
could have been differences in interpretation of this question. It was impossible to
reconcile data from different sources within the context of this programme evaluation. It
suffices here that directors reported as their total institutional funding between 19 and
331% of the funding level extracted from UNIDO records (and reported in Table 2.1).
The responses from directors thus deviated substantially from the management records, as
many directors reported lower total support budgets (up to five times lower) while some
reported higher total support budgets (up to 3.3 times higher).

There is thus a large spread between the total funding contributions made to different
countries (in the order of magnitude of the NCPC with the highest funding received at
least 5 times more than the NCPC with the lowest funding levels). Moreover, it should be
noted that the financial contribution to the NCPC does not relate to the size of the
economy or its structure. Or in other words, the funding commitment made to the NCPC
is not linked to the potential need or market for CP and CP-related services.

The reported annual budgets (i.e. resources at the disposal of the NCPC including
national government support, fee-for-service income, and other donor funding) vary
between USD 50,000 and USD 3.6 Million (data for 29 countries). This highest figure
(for Republic of Korea) is nearly three times higher than the second highest (Vietnam,
USD 1,333,000) and therefore excluded from calculation of the average annual budget.
For the remaining 28 countries the average annual budget is USD 438,000. The relative
shares of the various income sources are displayed in Figure 3.1. This is based on 35
centres that provided information on the sources of their income. The diamonds in the
figure show the average values for all NCPCs/NCPPs, and the error bars show the
variation between the highest and lowest. The average percentage contributions from
various sources are: 28.2 % for UNIDO-UNEP CP Programme; 26.2 % for private sector
(fee-for-service); 22.9 % for other donor programmes and 18.2 % for national

50
government. The contributions from other sources and other UNIDO projects are
negligible on average, but can still be substantive for some centres.

Figure 3.1: Sources of income (35 responses)

sources of income
(n=35)
100.0 100 100 100

90.0 90

80.0

70.0
% of total income

60.0

50.0

40.0
35
30.0
28.2
26.2
22.9
20.0 20
18.2

10.0

3 3.6
0.0 0 00.9 0 1 0
NCPC programme Other UNIDO Other Donor Government Private sector Other

average lowest highest

Figure 3.2: Staffing of the NCPCs/NCPPs (34 responses)

Staffing Levels
(N=34) (excl Tunisia and Paraguay)

50.0

45.0
44.0

40.0
39.0

35.0 35.0

30.0
# of FTE

25.0

20.0

17.0
15.0

11.3
10.0

7.0 6.9 7.0

5.5 5.9
5.0

1.9 2.5
1.0 1.0
0.0 0.0 0.0 0.0 0.0
total female total male total management professional support

average lowest highest

51
The variability in funding levels and annual budgets is reflected in the staff size of the
NCPCs/NCPPs. Data on staff were obtained from all centres, but the highest (Tunisia,
112 staff) and lowest Panama (no staff) were further excluded from the analysis. Data for
the remaining 34 responses are presented in Figure 3.2. The average for all centres is
represented by the squares, whereas the range bars point to the lowest and highest
numbers in this subset of centres. The average staff strength is 11.3 full time equivalent,
comprising 1.9 in management, 6.9 at professional level and 2.5 at administrative and
support levels. The gender balance is well attained, respectively on average 5.5 female
and 5.9 male staff members.

3.3 Service Delivery

In the first survey, the centres were requested to provide their current activity levels in
each of the five key service areas within the UNIDO-UNEP CP Programme, respectively:
information dissemination/awareness creation; training; in-plant assessments; policy
advice and transfer of Environmentally Sound Technologies (ESTs). The responses are
summarised in Table 3.5. It shows that three core services are very common in the
programme as they are delivered by at least 80% of the responding NCPCs/NCPPs, i.e.
information dissemination, training and in-plant assessments. The other two service
categories are less commonly delivered throughout the programme, respectively 56% of
the respondents is involved in policy advice and 47% in EST transfer. Furthermore, 36 %
of the responding NCPCs/NCPPs stated to be active in other service areas. The other
category is quite diverse, and includes e.g. Occupational Health and Safety,
environmental impact assessment, life cycle assessment and design for sustainability.

Table 3.5: Core service delivery (36 responses)

Service Category Number of Countries
Active Not active No Response Total
1. Information 29 81% 3 8% 4 11% 36 100%
Dissemination
2. Training 29 81% 3 8% 4 11% 36 100%
3. In-plant 29 81% 3 8% 4 11% 36 100%
Assessments
4. Policy Advice 20 56% 12 33% 4 11% 36 100%
5. EST transfer 17 47% 15 42% 4 11% 36 100%
6. Other 13 36% 18 50% 5 14% 36 100%

3.3.1 Potential for CP-related Service Delivery

The first part of the second survey addressed the potential for CP-related service delivery.
16 such areas were identified from among the topics covered by UNIDO under the term
‘CP Plus’, by UNEP under the term ‘SCP’ and donors under the term ‘CSR’. The
NCPCs/NCPPs were requested to assess the opportunity these service areas presented in
their countries, on the basis of their assessment of the potential for service delivery and
the perceived interest of key stakeholders in their countries. It was also requested to
identify whether and how they were active in regard to service delivery on these topics.
As the initial discussions with Centre Directors and the Steering Committee had revealed
a lack of common understanding on the meaning and scope of the different terms, an
attempt was made to define all 16 CP-related service areas, as per the following (18):
18
The umbrella terms (CSR, SCP and CP+) were purposely left out to avoid further confusion.

52
1. Energy Efficiency and Renewable Energy (EERE): application of CP methods, tools
and practices to increase energy productivity and use of renewable energy sources,
and reduce net greenhouse gas emissions;

2. Hazardous Waste Management (HWM): application of CP principles and practices to

reduce hazardous waste generation and achieve environmentally sound treatment
and/or disposal;

3. Eco-Industrial Parks/ Environmental Management of Industrial Estates (EIPs):

application of environmental best practices in planning, establishment and ongoing
management of industrial zones, estates and/or parks;

4. Life Cycle Assessment/ Management (LCA/M): methodology for assessing the

environmental impacts of products, services or processes considering all life cycle
stages;

5. Environmental Management Systems (EMS): planning, implementation, audit and

review of organisation’s effort to manage its environmental aspects in accordance
with its objectives and targets;

6. Environmental Management Accounting (EMA): use of materials and energy flow

data and associated costs in decision making;

7. Environmental Technology Assessment (EnTA): assessment of the environmental

aspects of alternative technologies (and/or the systems they are part off);

8. Financing CP/EST Investment Promotion (CP Finance): application of (advanced)

financing methods and investment promotion strategies for implementation of CP and
ESTs;

9. Sustainable Industrial Resource Management (SIRM): implementation of resource

efficiency/ dematerialisation and closed loop approaches in production chains;

10. Chemicals Leasing (CL): service oriented business model for provision of
chemicals/materials to industrial consumers;

11. Design for Sustainability/ Design for Environment/ Eco-Design (D4S): integration of
environmental (and possibly social) aspects into all aspects of product and service
development and delivery;

12. Sustainable Procurement/ Greening of Supply Chains (SusProc): inclusion of

environmental criteria in procurement of products and services by governments
and/or businesses;

13. Global Compact (GC): a set of overarching corporate responsibility codes to which
companies can make a voluntary commitment;

14. Triple Bottom Line/Sustainability Management (TBL): inclusion of environmental

and social dimensions into all aspects of (business) decision making;

53
15. Sustainability Development Reporting/Global Reporting Initiative (SDR): public
disclose of the organisation’s environmental, social and economic performance; and

16. Occupational Health & Safety/Labour Practices (OH&S): achieving a safe, clean and
productive workplace for all.

Figure 3.3: Estimated applicability of the service categories at the national level (23 responses)
applicability in country
(n = 23)

25

0 0 0 0
1 1 1 1
2 0 2 2 2 2
4
5 5 3
20 2 6
5 5
4
7 9 5
8
8 10
5 5
5 3
15
# of countries

13 5 9
8
11 5
10
10 6
9
18 12 10 10

14
13 5
12 12 6
5 10
8
7
6 6
5
4 4 4
3 3
0
EE&RE HWM EIP LCA/M EMS EMA EnTA CP SIRM CL D4S SusProc GC TBL SDR OH&S
Finance

high medium low don't know

Figure 3.4: Perceived interest of stakeholders at national level in the service areas (23 responses)
interest from stakeholders
(n=23)

25

0 0 0 0 0 0 0 0 0 0 0 0 0
1 1 1 1 1 1 1
0 2 2
4 2 4 4
3
20 6 6
5 5
6 4 8 6
8
7 9
12 6
15
# of countries

9 5
6 10 4
9
11 11
8
10
19 9
5
8 16
11 9
13
12 8
5
5 10
9 5
6 6 6
5
4
3 3
2 2 2
0
EE&RE HWM EIP LCA/M EMS EMA EnTA CP SIRM CL D4S SusProc GC TBL SDR OH&S
Finance

high medium low don't know no answer

The results in regard to estimated applicability of the service area and perceived interest
from stakeholders in the country are presented in Figures 3.3 and 3.4 respectively. These
figures show that:

¾There are five service areas that are commonly regarded applicable, as evidenced by
the fact that more than 90% of respondents estimated their potential as ‘high’ or

54
 ‘medium’. These are: Energy Efficiency and Renewable Energy (EERE), Hazardous
Waste Management (HWM), Environmental Management Systems (EMS),
Environmental Technology Assessment (EnTA) and Occupational Health & Safety
(OH&S). The evidence is strongest for EERA (rated as ‘high’ potential by 18
respondents and ‘medium’ by the remaining 5 respondents) and lowest for EnTA
(rated as ‘high’ potential by 8 respondents, and ‘medium’ by 13 other respondents).

¾Three service areas form a middle group as their potential is rated ‘high’ or ‘medium’
by at least 75% of the respondents. These are Environmental Management
Accounting (EMA), Cleaner Production Finance (CP Finance) and Sustainable
Industrial Resource Management (SIRM).

¾There is high uncertainty about the applicability of four service areas, as more than
20% of the respondents did not know-how applicable these would be in their home
countries. These are: Chemical Leasing (CL), Global Compact (GC), Sustainable
Development Reporting (SDR) and Eco-Industrial Parks (EIPs).

¾The perceived interest (in Figure 3.4) is an almost exact copy of the estimated
potential (in Figure 3.3). There are only minor changes, typically only 1 or 2
countries moved their response for stakeholder interest either one category higher or
one category lower than their rating of perceived applicability. The trend is that for
Life Cycle Assessment/Management (LCA/M) and Occupational Health and Safety
(OH&S) the perceived interest from stakeholders is somewhat higher than the
estimated potential. On the contrary, perceived interest from stakeholders appears to
be slightly lower than the estimated potential, for Hazardous Waste Management
(HWM), Eco-Industrial Parks (EIPs), Sustainable Industrial Resource Management
(SIRM), Chemical Leasing (CL), Design for Sustainability (D4S), Sustainable
Procurement (SusProc), Triple Bottom Line Management (RBL) and Sustainable
Development Reporting (SDR).

¾Some respondents added CP-related service delivery areas which they felt had
significant potential in their countries. These were: Corporate Social Responsibility
(CSR) (Kenya and Morocco), eco-labelling (Egypt), Profitable Environmental
Management (PREMA) (Egypt), Clean Development Mechanism (CDM) (Mexico),
Environmental Impact Assessment (EIA) (Morocco), chemicals management
(Slovakia) and EH&S legislation and compliance (Slovakia).

Overall there is thus general agreement about the perceived potential for service delivery
in areas that are focused on factories and technologies, i.e. Energy Efficiency and
Renewable Energy (EERE), Hazardous Waste Management (HWM), Environmental
Management Systems (EMS), Environmental Technology Assessment (EnTA) and
Occupational Health and Safety (OH&S). Many respondents have also commented in the
survey but also during the country visits that these have always been part of CP. There is
a high appreciation for the potential of Environmental Management Accounting (EMA),
CP Finance and Sustainable Industrial Resource Management (SIRM). The potential for
SIRM, however appears to be somewhat opportunistic or even misleading due to the
broad nature and appeal (or ‘jazziness’) of the term as two profound practical examples of
SIRM have been given a rather low rating (Eco-Industrial Parks (EIPS) and Chemicals
Leasing (CL).

55
Figure 3.5: Activity level of NCPCs in CP-related service delivery (23 responses)
activity level
(n=23)

25
23 23

21 21
20
20 19 19 19
18 18
17 17
16
15
15 14 14 14 14
# of countries

13 13
12 12 12 12 12
11
10 10 10
10 9 9 9

0
EE&RE HWM EIP LCA/M EMS EMA EnTA CP SIRM CL D4S SusProc GC TBL SDR OH&S
Finance

experience partners

Figure 3.6: Types of services delivered in CP-related areas (23 responses)

NCPC activity
(n=23)

25

20

15
# of countries

10

0
EE&RE HWM EIP LCA/M EMS EMA EnTA CP SIRM CL D4S SusProc GC TBL SDR OH&S
Finance

training pilot projects policy awards other

56
Figure 3.7: Partners for CP-related service delivery (23 responses).
NCPC Partners
(n=23)

25

20

15
# of countries

10

0
EE&RE HWM EIP LCA/M EMS EMA EnTA CP SIRM CL D4S SusProc GC TBL SDR OH&S
Finance

government industry ass prof ass UN agencies donors Public Interest Groups

The NCPCs were also requested to assess whether or not they are active in service
delivery and/or had established partnerships for service delivery in these areas. The
results are summarised in Figures 3.5, 3.6, and 3.7 and Table 3.6.

¾Figure 3.5 shows that 75% or more of the responding NCPCs had experience in
service delivery and most often also established partnerships in five of the CP-related
service areas, respectively: Energy Efficiency and Renewable Energy (EERE),
Hazardous Waste Minimisation (HWM), Environmental Management Systems
(EMS), Environmental Management Accounting (EMA), and Environmental
Technology Assessment (EnTA). Four of these (except EMA) were also the high
potential areas (as per Figures 3.3 and 3.4). Table 3.6 lists the countries that have
claimed experience and/or partnerships in the respective CP-related service areas.

¾Overall it appears that the NCPCs claim to have experience and established
partnerships in more service areas than which they perceive to have potential (as per
Figures 3.3 and 3.4), with the only exception for Occupational Health and Safety
(OH&S). This may be caused by opportunistic interpretations by the respondents of
what constitutes ‘experience’ and ‘partnerships’. It would appear that these topics are
touched upon in mainstream service delivery (e.g. CP assessments and policy
discussions) rather than made into separate service areas. To a certain extent, one
could therefore also interpret Figure 3.5 as a statement of expressions of interest for
service areas the NCPCs want to be in.

¾Figure 3.6 shows that training and capacity building is by far the most common way
in which the NCPCs are involved in the CP-related service areas, followed by pilot
projects/implementation. Only for EnTA there are more respondents indicating that
they are active in pilot projects than active in training.

¾Figure 3.7 shows a more diverse result in regard to partners for CP-related service
delivery. As a general observation it appears that NCPCs attempt to engage with
government and industry associations in their CP-related service delivery. For some

57
 topics they are slightly more focused on government (EERE, HWM, LCA/M,
SusProc and TBL) while for others they are slightly more focused on industry
associations (EIP, EMS, EMA, EnTA, CP finance, SIRM, CL, D4S, SDR and
OH&S). The involvement of UN agencies and donors is also very prominent,
evidencing that international cooperation and donor funding are an important catalyst
for extension of service delivery into CP-related fields.

3.3.2 Multilateral Environmental Agreements

The second part of the second survey concerned the activities of the NCPCs/NCPPs in
regards to implementation of MEAs. The Directors were asked to self-assess their level of
expertise (relative in the country) and indicate whether or not they were actually involved
at the national level (preparation of national strategies and action plans) or at the project
level (actual implementation in a company, city or otherwise). The five most applicable
MEAs in the CP area were included, respectively:
1. Johannesburg Plan of Implementation/Marrakech Process for Sustainable
Consumption and Production (SCP), further referred to as ‘Marrakech”;
2. Framework Convention on Climate Change, including Kyoto Protocol and Clean
Development Mechanism, further referred to as ‘Kyoto’;
3. Stockholm Convention on Persistent Organic Pollutants (POP), further referred to as
‘Stockholm’;
4. Basel Convention on Hazardous Waste Management, further referred to as ‘Basel’;
and
5. Montreal Protocol on Ozone Depleting Substances (ODS), further referred to as
‘Montreal’.
Two respondents replied to be involved in other MEAs, respectively the UNEP
International Declaration on Cleaner Production (Czech NCPC) and the European
Union’s REACH directive (Slovak NCPC).

The detailed results are provided in Figures 3.8 and 3.9, and Table 3.7. The following can
be concluded:

¾Figure 3.8 shows the result of the self evaluation of the NCPCs/NCPPs of their
expertise level on the different MEAs. The expertise level is highest for SCP
(Marrakech) for which just over 40% of the responding centres considers itself as a
leading expert in the country, with an additional 40% of the centres claiming to have
some expertise. The expertise level is lowest on ODS (Montreal).

¾Figure 3.9 shows that the activity level of NCPC is highest for SCP (Marrakech),
Climate Change (Kyoto) and POPs (Stockholm), with between 40 and 50% of the
responding NCPCs claiming to be involved in preparation of national plans and
strategies or development and implementation of specific projects (most commonly
with a particular company or business). The reported activity levels on HW (Basel)
and ODS (Montreal) are rather low with only a quarter of the responding

58
 Table 3.6: Experience and partnerships in CP-related service delivery (23 responses)
CP-related service Respondent countries with claimed Respondent countries with established
area experience partnership
1. Energy Bolivia, Cambodia, China, Colombia, Bolivia, Cambodia, China, Colombia,
Efficiency and Croatia, Czech Republic, Egypt, El Croatia, Czech Republic, Egypt, El
Renewable Salvador, Guatemala, India, Kenya, Laos, Salvador, Guatemala, India, Kenya, Laos,
Energy (EERE) Lebanon, Mexico, Morocco, Mozambique, Lebanon, Mexico, Morocco, Mozambique,
Nicaragua, Republic of Korea, Slovakia, Nicaragua, Republic of Korea, Slovakia,
Tanzania, Uzbekistan ,Vietnam and Tanzania, Uzbekistan, Vietnam and
Zimbabwe Zimbabwe
2. Hazardous Bolivia, Cambodia, Colombia, Czech Bolivia, Cambodia, Colombia, Czech
Waste Republic, Egypt, El Salvador, Guatemala, Republic, El Salvador, Guatemala, India,
Management India, Kenya, Laos, Lebanon, Mexico, Kenya, Laos, Lebanon, Mexico, Morocco,
(HWM) Morocco, Mozambique, Nicaragua, Mozambique, Nicaragua, Republic of
Republic of Korea, Slovakia, Tanzania, Korea, Tanzania, Uzbekistan, Vietnam and
Uzbekistan, Vietnam and Zimbabwe Zimbabwe
3. Eco-Industrial Cambodia, China, Colombia, Egypt, India, Cambodia, China, Colombia, Egypt, India,
Parks (EIPs) Lebanon, Mexico, Morocco, Mozambique, Lebanon, Mexico, Morocco, Mozambique,
Nicaragua and Republic of Korea Nicaragua and Republic of Korea,
4. Life Cycle Cambodia, Colombia, Czech Republic, Colombia, Czech Republic, Egypt, India,
Assessment/ Egypt, India, Kenya, Lebanon, Mexico, Kenya, Laos, Lebanon, Mexico,
Management Mozambique, Nicaragua, Republic of Mozambique, Nicaragua, Republic of
(LCA/M) Korea, Uzbekistan , Vietnam and Zimbabwe Korea, Uzbekistan, Vietnam and Zimbabwe
5. Environmental Cambodia, China, Colombia, Croatia, Czech Cambodia, China, Colombia, Croatia, Czech
Management Republic, Egypt, El Salvador, India, Kenya, Republic, Egypt, El Salvador, Guatemala,
Systems (EMS) Lebanon, Mexico, Morocco, Mozambique, India, Kenya, Lebanon, Mexico, Morocco,
Nicaragua, Republic of Korea, Slovakia, Mozambique, Nicaragua, Republic of
Tanzania, Uzbekistan, Vietnam and Korea, Slovakia, Tanzania, Uzbekistan
Zimbabwe ,Vietnam and Zimbabwe
6. Environmental Bolivia, Cambodia, Colombia, Croatia, Bolivia, Cambodia, Colombia, Croatia,
Management Czech Republic, Egypt, El Salvador, Czech Republic, Egypt, El Salvador,
Accounting Guatemala, India, Kenya, Lebanon, Mexico, Guatemala, India, Kenya, Lebanon, Mexico,
(EMA) Mozambique, Nicaragua, Republic of Mozambique, Nicaragua, Republic of
Korea, Slovakia, Uzbekistan, Vietnam and Korea, Uzbekistan, Vietnam and Zimbabwe
Zimbabwe
7. Environmental Bolivia, Cambodia, Colombia, Croatia, Bolivia, Cambodia, Colombia, Croatia,
Technology Czech Republic, Egypt, El Salvador, India, Czech Republic, Egypt, El Salvador, India,
Assessment Kenya, Laos, Mexico, Mozambique, Kenya, Laos, Mexico, Morocco,
(EnTA) Nicaragua, Republic of Korea, Tanzania, Mozambique, Nicaragua, Republic of
Uzbekistan and Vietnam Korea, Tanzania, Uzbekistan and Vietnam
8. Cleaner Bolivia, Cambodia, Colombia, Czech Bolivia, Colombia, Czech Republic, Egypt,
Production Republic, Egypt, El Salvador, Guatemala, El Salvador, India, Kenya, Lebanon,
Finance (CP Kenya, Lebanon, Mexico, Morocco, Mexico, Mozambique, Nicaragua, Slovakia,
Finance) Mozambique, Nicaragua, Slovakia, Tanzania, Vietnam and Zimbabwe
Tanzania, Vietnam and Zimbabwe
9. Sustainable Bolivia, Cambodia, Colombia, Czech Bolivia, Colombia, Czech Republic, Egypt,
Industrial Republic, Egypt, El Salvador, Guatemala, El Salvador, Guatemala, India, Kenya, Laos,
Resource India, Kenya, Laos, Mexico, Mozambique, Mexico, Mozambique, Nicaragua, Republic
Management Nicaragua, Republic of Korea, Uzbekistan, of Korea, Uzbekistan, Vietnam and
(SIRM) Vietnam and Zimbabwe Zimbabwe
10. Chemicals Cambodia, Colombia, Egypt, Guatemala, Cambodia, Colombia, Egypt, India, Kenya,
Leasing (CL) India, Kenya, Lebanon, Mexico, Nicaragua Lebanon, Mexico, Nicaragua and
and Uzbekistan Uzbekistan
11. Design for Cambodia, Colombia, Czech Republic, Cambodia, Colombia, Czech Republic,
Sustainability Egypt, Guatemala, India, Kenya, Mexico, Guatemala, India, Kenya, Mexico, Morocco,
(D4S) Morocco, Nicaragua, Republic of Korea, Nicaragua, Republic of Korea, Tanzania,
Tanzania, Vietnam and Zimbabwe Vietnam and Zimbabwe
12. Sustainable Bolivia, Cambodia, Colombia, Czech Bolivia, Cambodia, Colombia, Czech
Procurement Republic, Egypt, El Salvador, India, Kenya, Republic, Egypt, El Salvador, India, Kenya,
(SusProc) Mexico, Nicaragua, Republic of Korea and Mexico, Nicaragua, Republic of Korea and
Zimbabwe Zimbabwe
13. Global Compact Cambodia, Colombia, Egypt, India, Kenya, Cambodia, Colombia, Egypt, India, Kenya,
(GC) Mexico, Morocco, Tanzania, Vietnam and Mexico, Morocco, Vietnam and Zimbabwe
Zimbabwe
14. Triple Bottom Bolivia, Cambodia, Colombia, Croatia, Bolivia, Cambodia, Colombia, Croatia,
Line Egypt, El Salvador, Guatemala, India, Egypt, El Salvador, Guatemala, India,
Management Kenya, Mexico, Nicaragua and Zimbabwe Kenya, Mexico, Nicaragua and Zimbabwe
(TBL)

59
 CP-related service Respondent countries with claimed Respondent countries with established
area experience partnership
15. Sustainable Bolivia, Colombia, Egypt, India, Kenya, Bolivia, Colombia, Egypt, India, Kenya,
Development Mexico, Nicaragua, Republic of Korea and Mexico, Nicaragua, Republic of Korea,
Reporting Zimbabwe Vietnam and Zimbabwe
(SDR)
16. Occupational Bolivia, Cambodia, Colombia, Egypt, India, Bolivia, Cambodia, Colombia, Egypt,
Health and Kenya, Mexico, Morocco, Mozambique, Guatemala, India, Kenya, Mexico, Morocco,
Safety (OH&S) Nicaragua, Uzbekistan, Vietnam and Mozambique, Nicaragua, Uzbekistan,
Zimbabwe Vietnam and Zimbabwe

Figure 3.8: Self-evaluation of expertise level of NCPCs/NCPPs in regard to MEAs (23 responses)
NCPC expertise level
(n=23)

25

4
5
20
7
10
13
15
9
# countries

13
10 12

10
7
5 10

5
4
3 3
0
Marrakech Kyoto Stockholm Basel Montreal
Multilateral Environmental Agreements

leading expertise some expertise no expertise

Figure 3.9: Activity level of NCPCs in regard to implementation of MEAs (23 responses)
NCPC activity
(n=23)

25

20

15
# of countries

11 11
10
10 9

7
6 6

5 4 4

0
Marrakech Kyoto Stockholm Basel Montreal
Multilateral Environmental Agreement

national plans specific projects

NCPCs being involved in some form in implementation of these agreements.

Table 3.7 details which countries claim to be involved in activities for each of the
MEAs.

60
An opportunity was also provided for the NCPCs to detail their support needs. There
were only responses to this from 3 or 4 NCPCs on this for each MEA, and the support
needs were not specific, but rather generic for information materials and training (in
particular on consumption (for Marrakech) and Clean Development Mechanism (Kyoto)),
and support for project preparation (in particular for Kyoto and Stockholm).

Overall it can be concluded that the activity level of the responding NCPCs/NCPPs is
relatively modest. It would therefore appear that might be an opportunity for the Centres
to become more effective partners for their national governments and other stakeholders
for implementation of the various MEAs. In doing so, it should however be kept in mind
that typically the NCPC/NCPP is not the only institution in the country that is, or could
become, active on the various MEAs, as several countries have already set up dedicated
support structures for Kyoto and Montreal. It does appear that NCPCs/NCPPs can only
claim a degree of ‘exclusiveness’ in regard to the Marrakech process in that they are
typically the only, or at least one of the leading institutions on SCP in their home
countries.

Table 3.7: Experience of NCPCs in regard to implementation of MEAs (23 responses)

Multilateral Environmental NCPCs with activities in regard to NCPCs with project-related
Agreement development, implementation and activities for implementation of
or review of national strategies or MEA in specific companies,
action plans technologies or cities
1. Johannesburg Plan of Bolivia, Czech Republic, Egypt, Bolivia, Cambodia, Czech Republic,
Implementation/ Marrakech Kenya, Mexico, Mozambique, Egypt, El Salvador, Kenya, Mexico,
Process for Sustainable Nicaragua, Republic of Korea, Morocco, Mozambique, Republic of
Consumption and Production Slovakia, Tanzania and Zimbabwe Korea, and Vietnam
(SCP)
2. Framework Convention on Cambodia, India, Kenya, Morocco, Bolivia, Egypt, El Salvador,
Climate Change, including Nicaragua, Slovakia and Zimbabwe Guatemala, India, Nicaragua, Republic
Kyoto Protocol and Clean of Korea, Slovakia and Vietnam
Development Mechanism
(CDM)
3. Stockholm Convention on Cambodia, Croatia, Egypt, Guatemala, Bolivia, Guatemala, India and
Persistent Organic Pollutants Kenya, Morocco, Nicaragua, Slovakia Vietnam
(POPs) and Zimbabwe
4. Basel Convention on Hazardous Cambodia, Egypt, India and Bolivia, El Salvador, India, Slovakia
Waste Management Zimbabwe and Vietnam
5. Montreal Protocol on Ozone Cambodia, Egypt, Morocco, India and Morocco
Depleting Substances (ODS) Nicaragua, Tanzania and Zimbabwe

3.3.3 Resource Materials

The third part of the second survey dealt with current use and perceived usefulness of
selected resource materials. A listing of 16 resource materials was compiled at the
suggestion of project staff from UNIDO and UNEP from their recent offerings. These
were (19):
1. Cleaner Production Toolkit (UNIDO) (CD Rom) [40]
2. Training Kit on Cleaner Production Policy (UNIDO) (CD Rom) [41]
3. Chemical Leasing Business Models (UNIDO) (DVD) [42]
4. Energy Efficiency Guide for Industry in Asia (UNEP/SIDA) (web-supported CD
Rom [43]

19
Unfortunately the resource package on Responsible Enterpreneurship Achievement Programme (REAP) of UNIDO was
not brought to the attention of the evaluation team, as it would have been most appropriate to have included this CSR
package in this survey.

61
5. Energising Cleaner Production: a guide for trainers (UNEP/InWent Training
Package) [39]
6. Sustainable Consumption and Production: Making the Connection (UNEP Training
Package) [44]
7. Cleaner Production and Multilateral Environmental Agreements (UNEP Training
Package) [45]
8. The Application of Environmental Technology Assessment (UNEP/SIDA Training
Package) [46]
9. Advancing Sustainable Consumption in Asia: a guidance manual (UNEP-Asia ECO)
[47]
10. UNEP/IAPSO Product Criteria Database for Sustainable Public Procurement [48]
11. Design for Sustainability: a practical approach for developing economies
(UNEP/InWent) [38]
12. EcoDesign a Promising Approach to Sustainable Production and Consumption
(UNEP) [49]
13. Profiting from Cleaner Production; series of resource materials for raising capital and
finance for CP (UNEP) [36]
14. Capacity Building in Cleaner Production Centres; a training resource package
(UNEP) [34]
15. The Efficient Entrepreneur Calendar and Guidebook (UNEP and Wuppertal Institute)
[50]
16. Policy Instruments for Resource Efficiency: towards sustainable consumption and
production (UNEP CSRP) [51]

The results with regard to current use of these materials are presented in Figure 3.10. It
shows that only the CP toolkit is genuinely in common use in the NCPC/NCPP network
(21 current users), followed by the Capacity Building Package for CP Centres. Several
materials are also regularly used, in particular the Profiting from CP Package, the guide
on CP in MEAs, the Energising CP training package, EE Guide for Asia and the
SCP/connection guide. The two resource materials on Sustainable Consumption and
Sustainable Procurement are not yet used by any of the respondents, but as with other
resource materials there is good intent from several centres to start using them. However,
overall it is clear that there is potential for greater use of the resource materials. As was
also evidenced from

Figure 3.10: Current use of selected resource materials by NCPCs (23 respondents)
utilisation

Policy Instruments for RE 2 7 8

EE Calendar 6 5 6

Capacity Building in CPCs 15 1 3

Profiting from CP 9 6 2

Eco-Design Manual 5 8 2

D4S Guide 4 9 5

Criteria SusPro 0 6 10

Advancing SC in Asia 0 5 9

Application of EnTA 4 8 4

CP in MEAs 10 6 1

SCP: making connection 8 9 1

Energising CP (training) 7 9 2

EE Guide Industry AP 7 9 2

Chemical Leasing DVD 2 11 4

Training Kit CP Policy 10 7 1

CP Toolkit 21 1 0

0 5 10 15 20 25
# respondents (n=23)

already used planned to use unlikely to use

62
Figure 3:11: Perceived relevance of selected resource materials (23 respondents)
relevance

Policy Instruments for RE 1 3 2 1

EE Calendar 3 3 6 1

Capacity Building in CPCs 6 5 6 0

Profiting from CP 3 8 4 0

Eco-Design Manual 0 7 5 0

D4S Guide 1 4 7 0

Criteria SusPro 0 2 2 1

Advancing SC in Asia 0 1 4 0

Application of EnTA 0 5 6 1

CP in MEAs 3 7 5 0

SCP: making connection 1 9 2 0

Energising CP (training) 5 3 5 0

EE Guide Industry AP 4 7 3 0

Chemical Leasing DVD 0 5 4 1

Training Kit CP Policy 2 12 3 0

CP Toolkit 11 8 2 0

0 5 10 15 20 25
# NCPCs (n=23)

excellent high moderate low

the informal feed back on the survey (e.g. requests for access to electronic or hard copies
of resource materials), the resource materials appear to be not generally known within the
NCPC/NCPP network.

The quality of the resource materials was also surveyed, respectively in terms of
relevance of the contents, user-friendliness of the presentation and overall usefulness.

¾Figure 3.11 shows the results on relevance of the contents. The relevance is generally
considered good, as is evidenced by the fact that the majority of publications received
either an ‘excellent’ or ‘high’ rating on relevance from at least 50% of the
respondents, with as a very positive example the CP toolkit (rated ‘high’ or
‘excellent’ by over 90% of the respondents). The only publications with lower
relevance (i.e. less than 50% of respondents rating it ‘high’ or ‘excellent’), are the EE
Entrepreneur Calendar, the D4S Guide, the Sustainable Procurement Criteria,
Sustainable Consumption Guide and the EnTA training package.

¾Figure 3.12 shows the results on user-friendliness of resource materials, i.e. style,
modular design, presentation etc. The trend is very similar, suggesting that
perceptions of relevance and user-friendliness match reasonably well, albeit with a
slight tendency to rate some of the materials slightly lower on user-friendliness than
on content (e.g. the CP toolkit).

63
Figure 3.12: Perceived user-friendliness of selected resource materials (23 respondents)
User-friendliness

Policy Instruments for RE 1 3 2 1

EE Calendar 3 3 5 2

Capacity Building in CPCs 5 4 8 0

Profiting from CP 3 7 5 0

Eco-Design Manual 0 6 5 0

D4S Guide 0 4 8 0

Criteria SusPro 0 2 2 1

Advancing SC in Asia 0 1 4 0

Application of EnTA 0 3 6 2

CP in MEAs 2 6 6 0

SCP: making connection 1 6 4 0

Energising CP (training) 3 4 5 0

EE Guide Industry AP 3 6 4 0

Chemical Leasing DVD 0 4 5 1

Training Kit CP Policy 2 10 4 0

CP Toolkit 8 8 5 0

0 5 10 15 20 25
# of NCPCs (n=23)

excellent high moderate low

Figure 3.13: Overall usefulness of selected resource materials (23 respondents)

overall usefullness

Policy Instruments for RE 0 4 2 1

EE Calendar 2 4 5 2

Capacity Building in CPCs 5 6 6 0

Profiting from CP 3 7 5 0

Eco-Design Manual 0 6 5 0

D4S Guide 0 4 8 0

Criteria SusPro 0 2 2 1

Advancing SC in Asia 0 1 4 0

Application of EnTA 0 3 6 2

CP in MEAs 2 5 7 0

SCP: making connection 2 6 3 0

Energising CP (training) 4 3 5 0

EE Guide Industry AP 3 3 3 0

Chemical Leasing DVD 0 4 6 0

Training Kit CP Policy 2 11 3 0

CP Toolkit 9 9 3 0

0 5 10 15 20 25
# of NCPCs (n=23)

excellent high moderate low

¾In terms of overall usefulness (Figure 3.13), the general trend is that the majority of
respondents regard the materials as either ‘moderately’ or ‘highly’ useful. There are
positive exceptions, most notably the CP toolkit, which is considered ‘extremely
useful’ by 43% of respondents, and the Energising CP training and EE guide for AP,
both considered ‘useful’ by over 25 % of the respondents. The fact however that none
of the respondents considered seven (of the total of 16) resource materials ‘extremely
useful’, is of some concern, as it suggest that these publications do not address the
needs and opportunities of the NCPCs.

64
The second survey provided an opportunity for general feed back and requests. Most of
the open answers referred back to intents to use specific materials or start CP-related
service areas, confirming answers to earlier parts of the survey. Two overarching
comments were made, respectively to improve information flows within the NCPC
network (to ensure that NCPCs are aware of new initiatives and materials, and exchange
information between NCPCs) and a requests for training of NCPCs in new service areas,
for example on MEAs, funding mechanisms (including CDM) and product design and
consumption issues.

3.4 Self Assessment

The first survey invited the Directors to rate the expertise of their NCPC/NCPP in the
national context. Four major expertise areas were distinguished: cleaner production (20),
industrial environmental management (21), environment and industry policy (22) and
corporate sustainability (23). The results are presented in Table 3.8 (24). Over 80% of the
responding NCPCs/NCPPs claim to be a ‘leading expert’ in CP in the national context.
For the other expertise areas, the result is less outspoken. For industrial environmental
management and environment and industry policy, just over half of the respondents rate
their respective NCPC/NCPP as having ‘some expertise’, and just under 1/3 as being a
‘leading expert’ in the respective area. In case of corporate sustainability, 2/3 of the
NCPC claims to have ‘some expertise’, while the remainder split quite even between
either having ‘leading expertise’ or ‘no expertise’ in this area. Even though the result
should be interpreted with some care, as the self-evaluations could not be verified with
national stakeholders, it is apparent that the NCPCs/NCPPs feel confident being one of
the leading sources of CP expertise in their respective countries, while also being familiar
with related topics in industrial environmental management and environment and industry
policy.

Table 3.8: Self evaluation of key expertise areas (36 responses)

Expertise Areas Expertise Level of NCPC/NCPP
Leading Some No Unknown/ No Total
expertise Expertise Expertise Response
1. Cleaner Production 29 81% 5 14% 0 0% 2 6% 36 100%
2. Industrial 12 33% 21 58% 1 3% 2 6% 36 100%
Environmental
Management
3. Environment and 11 31% 20 55% 3 8% 2 6% 36 100%
Industry Policy
4. Corporate 5 14% 24 67% 5 14% 2 6% 36 100%
Sustainability

20
Described as: “process-integrated improvements in resource productivity and environmental performance”
21
Described as “environmental management accounting, environmental management systems, environmental and
sustainability reporting, life cycle assessment, eco-design, environmental labelling, closed loop systems”
22
Described as “e.g. stewardship, producer responsibility, Clean Development Mechanism, etc”
23
Described as “corporate social responsibility, global compact”
24
This evaluation is based on know-how and expertise, which complements the evaluation by the independent evaluators
based on institutional strength, as displayed in Table 2.2.

65
The final part of the self evaluation solicited a response from the NCPC/NCPP Directors
on the performance of their Centre/Programme against the evaluation criteria set for this
programme evaluation, namely (25):

1. Relevance: do businesses and other organisations in the country derive a benefit from
the Cleaner Production programme?

2. Effectiveness: are the services offered by the Centre and through the UNIDO
programme useful for implementation of Cleaner Production?

3. Efficiency; does service delivery through the Centre and UNIDO Programme make
best use of available resources?

4. Sustainability: is it likely that the benefits from the Centre and UNIDO Programme
will continue into the future? and

5. Ownership: to what extent are local stakeholders (industry, government, etc)

contributing resources to implementation of Cleaner Production and/or operation of
the Centre.

The results are presented in Table 3.9. The table displays a high level of confidence from
the Directors that their NCPC/NCPP performs quite well across the board, in particular if
the no responses are taken out of the comparison. The self evaluation is most optimistic
about relevance and effectiveness, rated ‘high’ by respectively 67% and 61% of the
respondents and rated ‘medium’ by respectively 19% and 22% of the respondents. The
assessment is still good for efficiency, rated ‘high’ by 50% of the respondents and
‘medium’ by 25% of the respondents. It would appear that there is some more doubt about
performance against sustainability and ownership, with the ‘high’ self-evaluations falling
to respectively 39% and 28% of the respondents and the ‘medium’ ones increasing to
respectively 36% and 39%.

Table 3.9: Self evaluation against evaluation criteria (36 responses)

Evaluation Self Assessment Rating
Criteria High Medium Low Unknown or Total
No response
1. Relevance 24 67% 7 19% 1 3% 4 11% 36 100%
2. Effectiveness 22 61% 8 22% 0 0% 6 17% 36 100%
3. Efficiency 18 50% 9 25% 2 6% 7 19% 36 100%
4. Sustainability 14 39% 13 36% 3 8% 6 17% 36 100%
5. Ownership 10 28% 14 39% 8 22% 4 11% 36 100%

25
After the survey instrument was distributed, the evaluation criteria were slightly adjusted and a sixth added (capacity
building).

66
4
Independent Country Evaluations
_______________________________________

4.1 Introduction

The findings from the third ‘pillar’ for the programme evaluation are summarised in this
chapter. Independent evaluation missions were undertaken to obtain first hand
information from the Director and staff of selected NCPCs, members of their boards,
national government agencies, industry associations, clients of NCPC services (including
former trainees, audited companies and other collaborators). Other initiatives not directly
associated with the NCPC but with a role in CP and/or related fields in the country were
also considered. The respective visit schedules were organised by a national consultant
under the direction of a member of the international evaluation team and in consultation
with the NCPC. The international and national consultant then spent some 2 to 5 working
days in the country to undertake semi-structured interviews with the nominated
representatives of the selected organisations. A detailed country review report was then
prepared by the international consultant with substantive input form the national
consultant. This contains a comprehensive analysis of arrangements in the preparation
and operation stages of the NCPC, participation of the NCPC in the global UNIDO-
UNEP CP Programme, detailed analysis of results achieved in each of the main service
categories and a country level assessment against the programme evaluation criteria.
Moreover specific recommendations were made for the further development of the
respective NCPC in its specific national context. These detailed country evaluation
reports will be available on request from the UNIDO Evaluation Group.

Resource constraints to complete this programme evaluation within the available budget
and within reasonable timeframes meant that only one member of the international
evaluation team could undertake each evaluation mission. The diverse backgrounds and
experiences of the team members and the need to undertake any evaluation with a
reasonable degree of professional judgement have introduced some variability between
the sets of independent country evaluations undertaken by the four international
evaluators. The comparison of country level findings is therefore bound to limitations,
and this is herewith explicitly acknowledged by the evaluation team. As the differences
between countries from the different regions and within these regions are already very
considerable, no further attempt was made to ascertain whether an evaluator-bias exists in
the evaluation results.

This chapter provides a summary of the 18 country review reports prepared for this
programme evaluation. It focuses on key issues and trends that emerged from these
independent country evaluations and that are important and relevant for the future of the
UNIDO-UNEP CP Programme (rather than just relevant within the respective country).
In so doing, this summary chapter does in no way justice to the richness of analysis and

67
evaluation that has been performed at the country level. The reader is therefore
encouraged to access the additional details in the respective country evaluation reports.

This chapter has been structured in four main sections. Section 4.2 provides a justification
for the selection of countries for which an independent evaluation was undertaken.
Section 4.3 then provides a qualitative summary of key issues identified in regard to
preparation and operation stages of the NCPC, and its participation in the global UNIDO-
UNEP CP Programme. Next, sections 4.4 and 4.5 provide a semi-quantitative summary
respectively of the comprehensive analyses of the results achieved by the visited NCPCs
(section 4.4) and of the detailed country level assessments by the evaluator against the six
evaluation criteria for this programme evaluation (section 4.5).

4.2 Country Selection

The selection of countries for the detailed independent evaluation was an iterative process
within the evaluation team, and then with the Steering Committee to arrive at the final
list.

In the first instance a rough cut was made of countries that needed to be either included or
excluded. At the request of the Government of Switzerland five countries were included
that needed in any case an evaluation as part of their funding cycle. These were:
Colombia, Costa Rica, El Salvador, Guatemala and Nicaragua. Moreover it was decided
to exclude the countries where only a NCPP is in operation, i.e. for which a decision has
not yet been made as to whether a full NCPC will be set up. This excluded Armenia,
Cambodia and Laos.

This left 30 countries from which 14 could be selected for inclusion on the list of
countries to be visited for an independent evaluation. This selection was approached with
a view to achieve maximum diversity within the subset of selected countries on a range of
characteristics of both the country as well as the NCPC, in particular:

ƒ Geographically: inclusion of approximately half of the NCPCs in each of the five

regions in which the programme operates (respectively: Africa (10 NCPCs), Asia (9
NCPCs/NCPPs), Central America (8 NCPCs), South America (4 NCPCs) and Central
and Eastern Europe (7 NCPCs/NCPPs), and within each region a reasonable
distribution geographically and socio-economically.

ƒ Donors: diversity of donors in order to include also some NCPCs which are funded
by donors who make a relatively smaller contribution to the programme (respectively
(Mozambique (funded by Italy), Sri Lanka (funded by Norway), Croatia (funded by
Czech Republic and Hungary), Kenya (funded by UNDP and Sweden)) or are no
longer funding the programme (The Netherlands as main donor for establishment of
the first generation of NCPCs in India, China and Mexico). It was also decided to
include countries in which a NCPC-like centre had been funded by one of the main
donors, but not through the UNIDO-UNEP CP Programme (the NCPCs in Colombia
and Peru which are directly funded by SECO).

ƒ Maturity: inclusion of NCPCs from first and subsequent generations. This

automatically resulted in the inclusion of some NCPCs which have been operating for
at least several years without institutional funding from the UNIDO-UNEP CP
Programme.

68
ƒ Size (of national economy and contribution of industry): a reasonable distribution of
NCPCs in large, medium and small countries, and within those some diversity in
regard to the level of development of the manufacturing sector.

Upon a number of iterations the final selection of 19 countries was confirmed. Each
country included in the list can be justified, as it might also have been possible to argue
individually for each of the not-selected countries that they should have been in the
shortlist. Doing so goes however beyond the scope of this programme evaluation. It
suffices here to demonstrate that the list of selected countries does meet the objective of
being diverse and inclusive, as per the above criteria.

Table 4.1: Visit list for independent evaluations by region

Region [selected/all Visit List [evaluator, year of establishment] (*) Non-Visit List
NCPCs/NCPPs]
Africa [5/10] Egypt [HS, 2004], Kenya [HS, 2000], Morocco [MM, Ethiopia, Tanzania,
2000] Mozambique [RvB, 2001] and South Africa Tunisia, Uganda and
[RvB, 2002] Zimbabwe
Asia [5/9] China [RvB, 1995], India [RvB, 1995], Sri Lanka Cambodia, Laos and
[RvB, 2002], Uzbekistan [HS, 2002] and Vietnam Republic of Korea
[RvB, 1998]
Central America [5/8] Costa Rica [MM, 1999], El Salvador [MM, 1999], Cuba, Honduras and
Guatemala [MM, 1999], Mexico [MM, 1995] and Paraguay
Nicaragua [MM, 1999]
Central and Eastern Croatia [HS, 2000] and Slovakia [HS, 1995] Armenia, Czech
Europe [2/7] Republic, Hungary and
Russia (**),
South America [2/4] Columbia [JD, 1998] and Peru [JD, 2001] Bolivia and Brazil,
(*) HS = Hans Schnitzer, JD = Johannes Dobinger, MM = Mathias Meyer and RvB = Rene van Berkel
(**) Russia has a regional CP Centre (in St Petersburg) and a sector specific CP Centre (in Moscow for the oil
and gas industry). Both operate independently within the UNIDO-UNEP CP Programme

Table 4.1 contains the country list by region, along with the date of establishment of the
NCPCs on the visit list and the evaluator who undertook the respective independent
country evaluation. Exactly half of the NCPCs in Africa and South America were
included, whilst Central and Eastern Europe was somewhat under-represented and both
Asia and Central America slightly over-represented. In terms of maturity, four of the
NCPCs on the visit list were established in 1995, two in 1998, four in 1999, three in 2000,
two in 2001, three in 2002 and one in 2004. This compares reasonably well with the
establishment history of the NCPCs as summarised in Table 3.2.

Table 4.2 provides a matrix listing of the host countries of the NCPCs/NCPPs by their
level of industrialisation (measured by Manufacturing Value Added (MVA) per head of
population) and total size of their economy (measured by their absolute Gross Domestic
Product). Data are for 2005 from internal sources in UNIDO and using standard UNIDO
categories. The countries on the visit list are underlined. The distributions are reasonably
good by column and by row, even though not all matrix cells are represented in the visit
list. Overall, ‘medium' level industrialised countries are somewhat over-represented in the
visit list with marginal under-representations for the ‘low’ and ‘extremely low’ levels of
industrialisation. Also ‘medium’ and ‘big-sized’ economies are slightly over-represented
with an under-representation of the ‘small sized’ economies.

69
Table 4.2: Host countries for NCPCs/NCPPs by level of industrialisation and total size of
economy (based on UNIDO internal data) (underlined countries have been visited for an
independent evaluation).
Level of Industrialisation (on basis of per capita MVA)
Extremely Low Low Medium High
[4/9] [5/11] [8/12] [2/5]
Small [6/16] Cambodia Armenia Costa Rica
Ethiopia Bolivia El Salvador
Kenya Honduras Uzbekistan
Laos Paraguay
Mozambique Zimbabwe
Nicaragua
Tanzania
Uganda
Size of Medium Cuba Lebanon Croatia
Economy [8/13] Egypt Peru Hungary
(absolute Guatemala Tunisia Korea
GDP) Morocco Slovakia
Sri Lanka
Vietnam
Big [2/3] Columbia Czech
South Africa Republic
Very Big [3/5] India Brazil
China
Mexico
Russia

In light of the limited number of countries and the very different socio-economic, size,
location and maturity criteria, it is concluded from Tables 4.1 and 4.2 that the visit list is
illustrative for the total set of host countries. The selection was however NOT
RANDOMISED which essentially means that the results for the visited countries CAN
NOT BE EXTRAPOLATED to the set of all NCPCs. However, as detailed in the
remainder of this chapter, each NCPC was found to be largely unique in its combination
of activities, results and organisational and institutional set up, which would have meant
that even with randomised country selection meaningful extrapolation may not have been
possible.

Unfortunately one country dropped out on the basis of the country visit, as in Slovakia it
turned out that the NCPC is no longer significantly involved in public interest advocacy
for CP, and therefore difficult to compare with the other NCPCs. No detailed country
evaluation could therefore be prepared as it was felt inappropriate to prepare an
evaluation of a private consulting company. The drop out of Slovakia, meant that the
quality of the remaining list of 18 visited countries worsened in particular in regards to
regional representation (as per Table 4.1) as only one of the seven NCPCs in Central and
Eastern Europe remained on this visit list for the detailed evaluation. However, due to the
given time-line of this programme evaluation, it was not possible to make adjustments to
the country selection. To a certain extent however, a case could be made that Uzbekistan
could have been added to this group, as Uzbekistan may have more in common with the
former Soviet-type of planning economies than with the rest of Asia. The Slovakia
example however proofs that the UNIDO-UNEP CP Programme is focusing on
developing countries, and as these countries reach higher levels of industrialisation the
NCPC will change its service portfolio, governance and operations.

70
4.3 National Implementation

This section discusses findings from the reviews of the activities undertaken to prepare,
establish and operate the NCPC in the host country. It focuses on findings that are of
significance at the programme level (not only in the specific national context). The
summary is organised in three main clusters, respectively pertaining to preparation stage
for the NCPC (paragraph 4.3.1), pertaining to the operational stage of the NCPC
(paragraph 4.3.2) and concerning participation of the NCPCs in global programme
activities (section 4.3.3).

4.3.1 Preparatory Stage

The independent country evaluations reviewed the preparatory activities and strategic
planning which were undertaken by, or on behalf of, the programme management prior to
the establishment of the respective NCPC. It should be noted however that this
programme evaluation did not attempt to revisit the pre-establishment stages for the
visited NCPCs in great detail as many had been operating for five years or more, and it
was therefore difficult to assess properly what had been done during the pre-
establishment phase and confirm its appropriateness in the circumstances that prevailed at
that time. Two aspects are of critical importance for the quality of the preparatory
activities, i.e. justification and feasibility.

The project justification is expected to confirm that CP is relevant, timely, applicable and
valuable to industry and government, and ascertain that establishment of a NCPC is an
appropriate mechanism for fostering the uptake of CP in the country. From a programme
perspective this national justification can also be interpreted as country selection. A
number of common issues appeared in several of the reviewed countries, including:

ƒ For the first five NCPCs, established in 1994-1995, no country specific justification
was undertaken (China, India, Mexico, Tanzania and Zimbabwe). These were
established under a common multi-country project agreement that was justified in the
context of the Agenda 21 commitment of industrialised countries to assist developing
countries with capacity building for and implementation of CP. The countries were
selected following an open call for expressions of interest, and this de-facto
substituted very well for country specific justifications (see also paragraph 2.3.1).
Interested countries had to apply to have a NCPC established and those with the best
applications were selected by the programme management. Automatically these were
the countries that had the best understanding of how CP could help their respective
country’s development.

ƒ Later on the programme implementation model changed and host countries for new
NCPCs were essentially decided upon in principle between the host country, a donor
country willing to provide in principle support and the programme management
(UNIDO, nationally and/or at headquarter level). The project documentation was then
prepared with project justification being a formality for signing off the project
agreement rather than an in-depth analysis of the country context and needs for CP.
While this is understandable in light of the systemic constraints faced by programme
management (see also Chapter 2), in most cases this resulted in fairly generic
justification statements, providing some data on the severity of industrial
environmental pollution, and arguing that industry faced challenges in a globalising

71
 economy and that CP was aligned with MEAs that the host country had committed to.
These statements, while correct in principle, do not demonstrate that CP is the right
intervention, i.e. that industrial environmental pollution was being recognised as a
national priority and that the target industries would be able to implement CP and
achieve benefit from so doing.

ƒ In several countries, the project justification was strengthened by referring back to the
success of earlier CP demonstration projects (South Africa, India, China, Sri Lanka,
etc.), making the assumption that because some companies were able to implement
CP as part of such demonstration projects, a majority of industries in the country
would be able to do so (which at least is questionable due to the self-selection bias for
environmentally motivated companies to participate in CP demonstration projects).
While in some of such countries, the NCPC project then retained the capacity created
with such earlier demonstration projects (e.g. China, India, Vietnam and Sri Lanka) in
other countries the NCPC project set out to build new capacity in parallel to existing
CP capacity created under earlier projects (e.g. South Africa).

Overall it appeared that project justification was approached as a formality that needed
completion prior to sign off of the project agreement, instead of an opportunity to assess
the national context, identify ways to harness any existing capacity, and target the NCPC
project to national socio-economic and environmental priorities. Prior to this programme
evaluation it had already been pointed out by several country level project evaluations
(e.g. [52, 53]) that this had resulted in project models and delivery strategies that did not
sufficiently address local circumstances.

There are also a few good examples in regard to project justification, for example Egypt,
Morocco and South Africa (the latter two in their second project period). In all of these,
the national government, either directly or in very close consultation with the host
institution and the private sector, took charge of justification and customisation of the
project model and strategy to existing national CP and related capacities. It should also be
noted that with the commencement of operations of the NCPC typically more information
on national context (legislation, economy, technology, etc.) and private and public sector
needs has become available, which then strengthened the justification for the NCPC.
Moreover, CP service delivery created CP examples and advocacy for CP-conducive
policy change, all of which contributed to clarifying the relevance of CP in the national
context and hence indirectly bolstering the justification for establishment of the NCPC.

Another key consideration in the project preparation is feasibility, i.e. the likelihood that
the project can be implemented as per the project agreement. A few trends appeared in the
visited countries:

ƒ Most project agreements attempted in one way or another to justify the creation of an
NCPC by claiming that on the medium term there would be a market for CP service
delivery that could underpin a financially-self sustaining NCPC. Throughout the
Programme these claims have been overtly opportunistic, and been lacking a reality
check (for example in regard to the size of the industry sector, existence of markets
for other business services, etc, etc.). This has been repeatedly pointed out in the
independent country evaluations done for this programme evaluation also in the
earlier programme [22, 23] and impact [25] evaluations as well as national
evaluations for several countries [26, 52, 53]. These over-estimations of the market
for CP services appear to be rooted in unrealistic expectations regarding the economic

72
 benefits from CP implementation (that CP would be a win-win proposition for all
businesses) and an over-estimation of the willingness of businesses to pay (in
particular in developing countries where many services to businesses are either highly
subsidised or free). Even the NCPCs themselves and representatives of their national
governments and industry associations in several cases expressed their beliefs that
there was no ground to justify claims in regard to the size of the market for CP
services. Over-optimistic appreciation of the existence and/or potential for
development of a CP market appears related to the generally supply-driven approach
for establishment of new NCPCs.

ƒ The initially lean project implementation model has been abandoned over time, as
current donors have been willing to invest considerably more on each NCPC than
initially envisioned when the Programme was conceptualised and launched (see Table
2.1 for specific data). There is a valid argument that supporting a NCPC with
substantial international expertise is helpful to position it as ‘THE’ leading institution
and that this could assist with long term survival of the NCPC. From the country
evaluations it did however appear that the downsides of this approach in terms of
overall feasibility of NCPC establishment have not been identified or no risk
management strategies put in place. Firstly, the NCPC develops a dependency on the
international experts. Even though the quality of the NCPC work might be
outstanding, there is no guarantee that such quality can be maintained if the hot-line
to the international consultants is no longer available or has to be factored into the
cost of local service delivery. Even though there is not yet any evidence for this (as
none of the higher funded NCPCs has yet had to transition to operation without
institutional funding support), there are challenges for several NCPCs, most urgently
in Vietnam. Secondly, as the Programme is based on a co-investment (cash and/or in
kind) from the host institution and/or its national government, the increase in donor
funding has upped the stakes for the national counterparts. In several countries this
has stretched the host institutions to make commitments for in kind and/or cash
commitments to the operation of the NCPC. The programme management appeared
not to have procedures in place to ascertain whether or not it was realistic to expect
that the host institution could meet such commitments. In case of Sri Lanka and
Mozambique for example the commitments could not be met as they were beyond the
means of the respective host institutions.

ƒ The UNIDO-UNEP CP Programme has been based on a host institution model that
establishes the NCPC in an existing institution, e.g. university, industry association,
public research institute or government agency. The CP centres in Colombia and Peru
evaluated here were however established as new institutions (similarly to earlier
programmes in the 1990s, such as those by the World Environment Centre, and the
US Environmental Pollution Prevention Programme). No clear evidence emerged
from the country evaluations to favour either institutional model or a particular type
of host institutions, as for each institutional set up there are countries with positive
and countries with negative experiences. As an overarching observation it can
however be pointed out that none of the project agreements appeared to have
undertaken serious risk analysis and management in regard to the host institution
arrangements. For example, working out ‘what if’ scenarios, in case the host
institution would not meet its commitments, would bail out completely, or would
cease to exist. This may not be a serious concern when the NCPC is hosted by a
major well-established national institution (university, research institute or alike) but
certainly deserved more attention where NCPCs are set up in small NGOs (e.g.

73
 Mozambique), industry associations (e.g. Guatemala) or within another donor-funded
project (e.g. Sri Lanka).

The above concerns in regard to feasibility assessments as part of project preparation re-
confirm the findings from the review on project justifications. Preparation for new
NCPCs has been approached with an emphasis on the fund-raising perspective, and once
a donor had in principle committed funds, project preparation mainly meant reaching
consensus with the local stakeholders regarding the operational modalities for the future
NCPC.

4.3.2 Operational Stage

The independent country evaluations considered the Programme’s approach to support

the NCPCs in establishing themselves as professional CP service delivery institutions.
The following key points appeared in several countries.

ƒ The project documentation for the NCPCs normally includes some provision for a
governance structure, most commonly a combination of a smaller management or
governing board, with decision-making powers, and a larger advisory board, with just
advisory capacities. The evaluation found that in most cases the governance
arrangements had been attended to and that these had to some extent contributed to
fostering local ownership in particular from national government. However it was
also found that governance could be significantly improved. Firstly, the importance of
governance appears to be underestimated and/or not sufficiently communicated in the
Programme. In several countries governance appeared to have been regarded by the
NCPC as a necessary condition for funding, and board structures were abandoned
shortly after the institutional funding to the NCPC through the UNIDO-UNEP CP
Programme ended (including e.g. China, India, Mexico and Croatia). Secondly,
several countries set up tripartite decision-making boards, comprised of
representatives of the donor government, the host country government and UNIDO.
Such tripartite boards do not invite input of key national stakeholders (e.g. industry
associations, NGOs, etc.), which reinforces a view that the board is a project
implementation mechanism rather than a mechanism to foster national ownership of
the NCPC and make its activities most relevant to various stakeholders (and hence
bolster the sustainability prospects of the NCPC). The meeting frequency of these
boards has been insufficient to provide timely and consistent guidance (once every 1
or even 2 years, whereas effective governance might be needed with e.g. a quarterly
frequency). Thirdly, the roles and decision-making protocols for the boards have not
been sufficiently clarified at the national level and guidance available at programme
level not implemented. For example at least the role of the NCPC director in the
board is problematic when it comes to decision making. As in the corporate world it
is by far preferred that at least the NCPC Director, but probably also the UNIDO and
donor representatives, have an ex officio role in the board, which is so far not the
case, and would then not have a vote in the board. Fourthly, in regard to advisory
boards it was found that attendance was in many countries reportedly low which
appeared to reflect a lack of interest and/or willingness on the part of sufficiently
senior representatives of key stakeholders to make board membership a priority.
Among the NCPCs there are some praiseworthy attempts to improve governance,
among the visited countries in particular in South Africa.

74
ƒ The programme management is to be applauded for emphasising the need for NCPCs
to develop and implement regular business plans, and providing training on business
plan development. There remains however scope for improvement in the business
planning processes, as there remains a tendency among NCPCs to operate
opportunistically and drift in its mission. Even though it is commendable that the
NCPC retains some flexibility to respond to needs and opportunities as they arise
nationally, there is a need for greater discipline among NCPCs to focus their limited
resources in selected critical activities rather than spread these too thin about too
many activity and topic areas. It is particularly challenging to avoid such mission drift
when the market for CP services does not yet exist, and needs to be established
through concerted and strategic activities of the NCPC.

ƒ Several of the visited NCPCs provided samples of recent training materials,

assessment reports and/or publications. Their review as part of this programme
evaluation showed that even though their average standard is professionally
acceptable, there remain opportunities for standardisation and professionalization of
the service delivery, and hence potential for greater effectiveness of services and
efficiency of service delivery. For example, consistent use of logos and presentation
styles, consistent use of concepts and methods, maximum use of national success
stories in promotional material and similar reporting formats, etc. It appeared that
such professionalization opportunities have so far remained unnoticed. The set up of
management systems certified or otherwise, would be a good incentive to standardise
service delivery (as demonstrated in Vietnam where the NCPC achieved certification
on both ISO 9001 and 14001, respectively for its quality and environmental
management systems). It was also noted that some of the multi country projects, for
example those on integration of energy efficiency into core CP, enforced assessment
methods and presentation formats that were not consistent with the national models
used by some of the NCPCs who implemented these projects. This issue of
professionalization and standardisation deserves greater consideration at programme
management level, from UNIDO, UNEP and donors.

ƒ Many NCPCs have invested significant resources in training of CP auditors, advisors

or trainers (for example up to some 8,000 in China alone), and are now increasingly
using these external CP professionals for delivery of NCPC services (e.g. conducting
CP assessments in companies, etc.). The creation and utilisation of a cadre of CP
professionals is supported as a multiplier mechanism. This means that the NCPC
increasingly assumes a project management and quality control role (or exclusively,
as the case would be in for example South Africa). While this is in principle a
perfectly valid strategy option for a NCPC, in several countries there is now ground
for concern that with a fully, or near fully, outsourced model, the NCPC may weaken
its core CP capacities, and in the end limit its own ability to do proper quality control
over outsourced activities.

Overall it appears that the Programme’s performance in supporting NCPCs in the visited
countries during their institutionally funded operational stage was on average satisfactory.
The most tangible areas for improving support to NCPCs are: transparent and effective
governance structures; strengthened (‘tighter’) business planning protocols; and
standardisation and professionalization of service delivery.

75
4.3.3 Programme Participation

The NCPCs are supported through programme level activities carried out by, or on behalf
of UNIDO, UNEP and donors. At this global level the UNIDO-UNEP CP Programme
provides for overall programme management and administration, networking activities
and technical assistance inputs (including international experts, training opportunities and
information materials). The country evaluations canvassed the experiences of the visited
NCPCs and their national stakeholders in regard to these global programme activities.
The following overarching issues emerged.

ƒ There are no formal management arrangements that define the relation of the
UNIDO-UNEP CP Programme with the NCPCs which are no longer institutionally
funded through the Programme. The programme management is therefore not aware
of the activities and achievements of these NCPCs, while they remain advertised and
acknowledged as UNIDO-UNEP NCPCs. This has raised different issues, which have
been recognised by programme management and contributed to the decision to
undertake this programme evaluation. Firstly, the impression is being maintained that
these NCPCs are obliged to implement the UNIDO-UNEP CP programme strategy,
while the programme management has no means to entice these NCPCs to do so.
Secondly, the NCPCs go on to develop activities as per their own assessment of local
business opportunities, and this may no longer be consistent with the UNIDO-UNEP
CP Programme or even general UN Policy. There is a need for the Programme to
establish appropriate means to engage with NCPCs after their institutional funding
through the UNIDO-UNEP Programme has ended. Even though formerly funded
NCPCs recognise the importance and relevance of the Programme, many of these
NCPCs are not in close contact with the Programme and they do not perceive to
receive concrete and practical benefits from it, apart from the prestige associated with
membership of a UNIDO-UNEP network. This was encountered in all visited NCPCs
that are no longer funded, in some countries even quite strongly (e.g. China and
India).

ƒ There is widespread concern among funded and previously funded NCPCs that
administrative arrangements and funding disbursements are too time-consuming.
Even though most NCPCs manage to cope with these problems, often with support of
local UNIDO representatives, others have struggled and at times had to prepay Centre
expenditures from their private funds to keep the NCPC going.

ƒ In principle, there is appreciation in most countries for the initiatives of the

programme management to extend the scope of CP and introduce new services.
However, there is concern about donor-driven identification of potential service areas,
and insufficient endorsement by NCPCs for their further scoping and integration with
core CP service areas. Moreover, government representatives in several visited
countries expressed a strong desire for the NCPC to remain relatively narrowly
focused on plant level CP activities, as the job of fostering CP uptake is by far not yet
completed (e.g. China and India). This highlights the current absence of a provision in
the Programme to survey periodically CP, and CP-related, needs of NCPCs and their
national stakeholders, to inform and guide strategic developments in the Programme.

ƒ There are very high, but non-specific, expectations regarding networking, which
remain so far largely unmet. Positive developments are the LatinNet network of

76
 NCPC and related activities in Latin America (see Box 2.1), the regional multi-
country projects in Asia Pacific (e.g. GERIAP) and through the Central American
Environmental Committee. Some steps have also been taken in Africa, but follow up
has not been forthcoming. There is a general preference in particular from the NCPCs
that networking would work best when focused around specific initiatives.

ƒ The availability of international expert inputs to the different NCPCs has varied
greatly. While some NCPCs operated essentially without access to international
expert inputs, others had for substantive periods of time access to short term and/or
resident part or full time technical advisors. The quality of the expert inputs has
generally been good. Several NCPCs would however appreciate greater involvement
in selection of international experts and customising their Terms of Reference better
to their immediate needs. As evidenced by the self-evaluation surveys (section 3.3)
the resource materials produced by UNIDO and UNEP for use by the NCPC network
are also generally perceived as informative and useful. However on the flip side, the
self assessment had revealed that NCPCs/NCPPs are insufficiently aware of the
complete range of information and resource materials made available by UNEP and
UNIDO (see paragraph 3.3.3).

In the visited countries the NCPCs and their national stakeholders remain loyal to the
global UNIDO-UNEP CP Programme, and are in principle supportive of initiatives to
broaden and/or deepen the Programme with additional service areas, while also creating
more networking opportunities. There is however a strongly felt need to address concerns
with regard to efficiency of programme administration, and effectiveness of networking
through increased availability and intensity of networking opportunities within the
Programme.

4.4 National Results

The results achieved through the establishment and operation of the NCPCs in the visited
countries were reviewed, in each of the five core service areas distinguished in the
UNIDO-UNEP CP Programme, respectively: information dissemination/awareness
raising; training; in-plant demonstrations; policy advice and EST transfer. The available
information on results was considered at three levels, namely:
ƒ Result Level 1: Outputs: activities undertaken or delivered by the NCPC;
ƒ Result Level 2: Outcomes: activities of the direct customers of the NCPC; and
ƒ Result Level 3: Impacts: benefits for local industry and other stakeholders in the host
country.

Where meaningful, a further distinction was made between ‘leading’ and ‘lagging’
evidence. Leading evidence is prospective and refers to the presence of initiatives that
could result in the uptake of CP (for example the definition and planning of a training
programme in regards to target group, learning outcomes and topics; initiation of a
demonstration project; engineering design for a CP technology option). Lagging evidence
is retrospective and refers to completed initiatives that have contributed to the realisation
of CP (for example people trained or CP options identified).

77
4.4.1 Information Dissemination

Information dissemination is achieved by means of production and distribution of

information materials (booklets, flyers, websites, etc.) and delivery of awareness type of
seminars. The latter are typically done in collaboration with other organisations, for
example regional or national government agencies, professional or industry associations,
universities and/or other NGOs.

The diversity of information and awareness initiatives in the visited countries is quite
large. In the country-specific evaluation reports detailed comments are provided in
regards to the current status of information and awareness initiatives in the visited
countries. For this global programme level summary a comparative analysis was
performed of the scope and results of the information activities in the visited countries.
Semi-quantitative scales were therefore used, as in Table 4.3. The results achieved are
presented in Table 4.4, and Figure 4.1 presents the main analysis results graphically.

78
Table 4.3: Categories used in comparative analysis of NCPC results on information dissemination in the visited countries
Service Area 1: Information Dissemination/Awareness Raising
Level Scale of Results (*) Type of Evidence Strength of Evidence
Unknown Some Result Good Result Excellent Unavailable Leading Lagging Both Unavailable Weak Strong
Result Evidence Evidence
1. Outputs No 2 or less per 3-6 per year 7 or more per No evidence Systems in Quantitative Leading No evidence Some data Extensive
information year year available place to plan information and available available but data
available and deliver on number lagging not available,
activities, of activities comprehensive covering
and monitor and all
participation participation activities
levels levels
2. Outcomes No Less then 2 % Between 2 and More then 10% No evidence Systems in Quantitative Leading No evidence Some data Extensive
information of recipients 10% of of recipients available place to information and available available but data
available known to have recipients known to have monitor on share of lagging not available,
acted known to have acted follow up by participants comprehensive covering
acted recipients of undertaking all
information some CP activities
activity
3. Impacts No Less then 2 % Between 2 and More then 10% No evidence Systems in Quantitative Leading No evidence Some data Extensive
information of recipients 10% of of recipients available place to information and available available but data
available have achieved recipients have have achieved monitor on benefits lagging not available,
some CP achieved some some CP impacts achieved by comprehensive covering
implementation CP implementation achieved by participants all
implementation participants activities
in
information
activities
(*) Number of information or awareness initiatives organised by NCPC and/or information materials produced.
79

79
 80
Table 4.4: Findings from analysis of results for information dissemination
Refer Table 4.3 for explanation of the categories
Service Area 1: Information Dissemination
Level 1: Outputs Level 2: Outcomes Level 3: Impacts

Country Region Scale Evidence Type Evidence Strength Scale Evidence Type Evidence Strength Scale Evidence Type Evidence Strength
China Asia Good Both Weak Some Lagging Weak Unknown Unavailable Unavailable
Columbia S America Some Both Weak Some Lagging Weak Unknown Unavailable Unavailable
Costa Rica C America Good Both Strong Some Lagging Weak Some Lagging Weak
Croatia CE Europe Good Both Strong Some Unavailable Weak Unknown Unavailable Unavailable
Egypt Africa Good Both Strong Good Lagging Strong Some Lagging Unavailable
El Salvador C America Good Both Strong Some Lagging Weak Some Lagging Weak
Guatemala C America Excellent Both Strong Good Lagging Weak Unknown Unavailable Unavailable
India Asia Good Leading Weak Some Lagging Weak Unknown Unavailable Unavailable
Kenya Africa Good Leading Weak Some Lagging Weak Some Both Unavailable
Mexico C America Some Lagging Strong Some Lagging Weak Unknown Unavailable Unavailable
Morocco Africa Excellent Both Strong Good Lagging Weak Some Lagging Weak
Mozambique Africa Some Both Strong Some Lagging Weak Unknown Unavailable Unavailable
Nicaragua C America Excellent Both Strong Good Lagging Strong Some Lagging Weak
Peru S America Good Both Strong Unknown Unavailable Unavailable Unknown Unavailable Unavailable
South Africa Africa Good Both Weak Some Lagging Weak Unknown Unavailable Unavailable
Sri Lanka Asia Excellent Both Strong Some Lagging Weak Unknown Unavailable Unavailable
Uzbekistan Asia Excellent Both Strong Some Lagging Weak Unknown Unavailable Unavailable
Vietnam Asia Excellent Both Strong Some Lagging Weak Unknown Unavailable Unavailable
Note: evidence categories are: leading = leading evidence only, lagging = lagging evidence only, and both = leading and lagging evidence.

80
Figure 4.1: Comparative analysis of results in regards to information dissemination
Information/Awareness

100% 0
1
3
90%

80%

70%
12
countries (n=18)

60%
9 13

50%

40%

30%

20%
6 6

10% 4

0% 0 0
Level 1: Outputs Level 2: Outcomes Level 3: Impacts

excellent good some unknown

Figure 4.1 shows that there was good performance in information dissemination. The data
are most comprehensive at output level, showing that 6 countries achieved ‘excellent’
output levels and 9 countries ‘good’ output levels (as per the categories in Table 4.3).
Data at outcomes level are less comprehensive, but regardless it was found that 4
countries had ‘good’ outcome levels and 13 countries ‘some’ outcomes. Impacts however
could not be rated for most countries (12 counties), whereas for the remaining 6 only
‘some’ impacts could be confirmed.

Table 4.4 also provides a more detailed summary of the available evidence for results on
information dissemination. The evidence basis is strongest at output level, as most
countries (15 countries) had ‘leading’ and ‘lagging’ evidence, and in most cases (13
countries) this was rated ‘strong’. This means that these NCPCs have systems and
processes in place to prepare and deliver information events, and do maintain some kind
of database of participants. The evidence basis at outcome level is much more limited. It
was rated ‘weak’ for the majority of countries (15 countries) and was also limited to
‘lagging’ evidence for the majority (16 countries). This essentially means that the NCPC
is aware that some of the former participants in the information events have taken some
steps towards CP uptake (for example signed up for training, or requested a CP
assessment). However, such data are not routinely collected. At impact level, there is
hardly any firm evidence. But there is anecdotal information confirming that one or a few
former participants have gone on and become CP advocates or implemented CP in their
own organisations.

From this comparative analysis of results on information dissemination (as presented in

Table 4.4 and Figure 4.1) it is concluded that the majority (> 80%) of the visited NCPCs
have a good portfolio of information dissemination and awareness building activities.

81
They have demonstrated their capability for preparing and delivering information
materials and awareness sessions, and have established some systems for keeping records
on attendance levels etc. Collection of data on outcomes and impacts

82
Table 4.5: Categories used in comparative analysis of NCPC results on training in the visited countries
Service Area 2: CP Training
Level Scale of Results Type of Evidence Strength of Evidence
Unknown Some Result Good Result Excellent Unavailable Leading Lagging Both Unavailable Weak Strong
Result Evidence Evidence
1. Outputs No 2 or less 3-6 training 7 or more No evidence Systems in Quantitative Leading No evidence Some data Extensive
information training programmes training available place to plan information and available available but data
available programmes per year programmes and deliver on number lagging not available,
per year per year training, and of training comprehensive covering
monitor programs all
participation and activities
levels participation
levels
2. Outcomes No Less then 20 % Between 20 and More then 50% No evidence Systems in Quantitative Leading No evidence Some data Extensive
information of trainees are 50% of trainees of trainees are available place to information and available available but data
available known to be are known to be known to be monitor on share of lagging not available,
active in CP active in CP active in CP follow up trainees comprehensive covering
initiatives by which are all
former active in CP activities
trainees
3. Impacts No Less then 20 % Between 20 and More then 50% No evidence Systems in Quantitative Leading No evidence Some data Extensive
information of trainees have 50% of trainees of trainees have available place to information and available available but data
available achieved have achieved achieved or monitor on benefits lagging not available,
demonstrable or contributed contributed to impacts achieved by comprehensive covering
CP to demonstrable demonstrable achieved by former all
implementation CP CP former trainees activities
implementation implementation trainees
83

83
 84
Table 4.6: Findings from analysis of results for training
Refer Table 4.5 for explanation of the categories
Service Area 2: CP Training
Result Level 1: Outputs Results Level 2: Outcomes Result Level 3: Impacts
Country Region Scale Evidence Type Evidence Strength Scale Evidence Type Evidence Strength Scale Evidence Type Evidence Strength
China Asia Excellent Both Strong Excellent Both Weak Unknown Unavailable Unavailable
Columbia S America Unknown Lagging Weak Unknown Unavailable Unavailable Unknown Unavailable Unavailable
Costa Rica C America Good Both Strong Good Lagging Weak Good Lagging Weak
Croatia CE Europe Some Both Strong Some Unavailable Unavailable Unknown Unavailable Unavailable
Egypt Africa Some Leading Weak Some Lagging Weak Unknown Unavailable Unavailable
El Salvador C America Good Both Weak Good Lagging Weak Some Lagging Weak
Guatemala C America Good Both Strong Good Lagging Weak Some Lagging Weak
India Asia Unknown Leading Weak Some Lagging Weak Unknown Unavailable Unavailable
Kenya Africa Excellent Leading Weak Good Both Strong Some Unavailable Unavailable
Mexico C America Good Both Strong Some Lagging Weak Some Lagging Weak
Morocco Africa Some Both Strong Some Lagging Weak Some Lagging Weak
Mozambique Africa Some Both Strong Good Lagging Strong Unknown Unavailable Unavailable
Nicaragua C America Excellent Both Strong Excellent Lagging Strong Excellent Lagging Weak
Peru S America Unknown Leading Weak Some Lagging Weak Unknown Unavailable Unavailable
South Africa Africa Some Both Strong Some Lagging Strong Unknown Unavailable Unavailable
Sri Lanka Asia Unknown Lagging Strong Some Lagging Weak Unknown Unavailable Unavailable
Uzbekistan Asia Excellent Both Strong Some Lagging Weak Unknown Unavailable Unavailable
Vietnam Asia Excellent Both Strong Some Lagging Weak Unknown Unavailable Unavailable
Note: evidence categories are: leading = leading evidence only, lagging = lagging evidence only, and both = leading and lagging evidence.

84
from information and awareness activities is hardly done at all and those few countries
that have attempted it do so in a relatively un-systematic manner.

4.4.2 Training

NCPCs deliver training on CP and CP-related topics to various target groups. Target
groups include: technical staff and/or management representatives from companies, future
CP auditors (e.g. from consultancies, universities, technical institutes and/or government
agencies) and government representatives (legislators, policy makers, etc., at national,
regional and/or local level). CP concepts and assessment methods form the core of most
training programmes, while some NCPCs have complemented this with one-off specialist
training programmes, for example for specific industry sectors or on topics considered as
advanced (e.g. Environmental Management Systems, Life Cycle Assessment, etc.).

The training portfolios of the NCPCs in the visited countries are therefore rather different.
While for some NCPCs training is one of the core activities (e.g. China) in others training
is only delivered in support of other main activities, like company demonstration projects
(e.g. Mozambique). The country-specific evaluation reports provide a detailed summary
and review of the training activities in the visited countries. For this summary a
comparative analysis was performed of the scope and results of training activities in the
visited countries. Semi-quantitative scales were therefore used, as in Table 4.5. The
results achieved are presented in Table 4.6, and Figure 4.2 presents the main analysis
results graphically.

Figure 4.2: Comparative analysis of results in regards to training

Training

100%
1

90% 4

80%

70% 11
10
5
countries (n=18)

60%

50%

40% 4

30%
5 5
20%
5
10% 1
2
1
0%
Level 1: Outputs Level 2: Outcomes Level 3: Impacts

excellent good some unknown

Figure 4.2 shows a quite diverse picture in regard to the scale of the training results
among the visited NCPCs. At output level, there is an almost even split of the visited
countries between the four category levels, respectively: ‘excellent’ (5 countries), ‘good’
(4 countries), ‘some’ (5 countries) and ‘unknown’ (4 countries) (using the category

85
definitions as in Table 4.5). At outcome level, the distribution of countries was:
‘excellent’ (2 countries), ‘good’ (5 countries), ‘some’ (10 countries) and ‘unknown’ (1
country). The higher certainty for training outcomes (only 1 country ‘unknown’) than for
training outputs (4 countries ‘unknown’) is largely caused by a time factor. There was
insufficient information on recent training volume (to rank performance at output level),
whilst it could be confirmed that several former trainees now deliver CP services (which
justified a rating on training outcome with weak evidence). At impact level, information
was insufficient to rank most of the countries (11 countries therefore scored an
‘unknown’). Of the NCPCs for which some impact data could be derived, one was ranked
as ‘excellent’ result, one as ‘good’ result and five as ‘some’ result.

Table 4.6 provides more detail in regard to datasets available at the NCPCs to
demonstrate training results. The majority of visited NCPCs (12 countries) have diverse
records at the level of outputs, including both ‘leading’ and ‘lagging’ evidence, that
provides a ‘strong’ evidence basis that training is being prepared and delivered on a
routine basis. The evidence basis at outcome level is much weaker, as the majority of
visited NCPCs (12 countries) only have lagging and incomplete records, i.e. they might
know that some trainees are using their newly acquired CP skills (most commonly as
contractors to the NCPC for undertaking CP activities (e.g. CP assessments)), but are not
aware whether and how the other trainees have used their CP training. At impact level the
evidence basis is even weaker than at outcome level.

From the comparative analysis of training results (as presented in Table 4.6 and Figure
4.2) it is concluded that for half of the NCPCs training is a core activity in its own right
with a considerable and sustained level of outputs. For the other visited NCPCs training
appears to be more narrowly focused and delivered only in support of other core
activities. Despite the considerable training efforts, data on outcomes and impacts are
scarce, incomplete and irregularly maintained.

4.4.3 Demonstration

NCPCs assist companies and other organisations with the identification and evaluation of
CP options, through the execution of CP assessments. In the early stages of establishment
of a NCPC such CP assessments are all done as demonstration projects, i.e. with the
explicit aim to develop CP assessment capabilities (of NCPC staff and associated experts)
and develop success stories/business examples for the further promotion of CP in the
country. Over time, a greater share of CP assessments is expected to be done on a fee-for-
service basis. Likewise the CP assessments are then typically conducted as either full CP
assessments (i.e. comprehensive root source and cause analysis, quantified waste streams
and investment costs, savings and environmental benefits) or as walk-through CP
assessments (also quick scans, pre-assessment, rapid assessment or otherwise, with
limited option generation, and only qualitative analysis of likely costs and benefits). Some
NCPCs have focused their CP assessment services in a few priority sectors (typically 3 to
5 sectors, for the countries with a well established manufacturing sector, e.g. South
Africa, Vietnam, Morocco, Egypt, Colombia) while others have not been able to develop
and/or maintain a clear focus (typically in those countries with a more narrow
manufacturing basis, e.g. Sri Lanka, Mozambique). This is important, as there is a
widespread view that sector focus increases the probability of impact through replication
of well-demonstrated CP successes.

86
The CP assessment portfolios of the NCPCs in the visited countries are rather different,
and within each country a degree of differentiation occurred, with regard to for example
individual and collective approaches, consulting or coaching models for CP assessments,
and level and type of support after completion of the CP assessment. An observation that
applied to most of the countries, albeit to different degrees, is that consistency in CP
concepts and assessment methods can be improved, leading to more standardised service
delivery with greater replication and marketing potential of completed CP demonstrations
(and hence effectiveness and efficiency of CP service delivery). There is a strong case for
customised concepts and methods at the national level, so that CP is made most relevant
to national circumstances, and that these develop over time as the national context
changes (for example with the current revision of the scope of the ‘China CP enterprise
CP audit manual’ [54]). However, it is recognised that this creates a tension with the
desire to achieve uniformity at international level (which created some tension with the
energy efficiency projects as discussed in paragraph 4.3.3)).

87
88
Table 4.7: Categories used in comparative analysis of NCPC results on CP Assessments/demonstrations in the visited countries
Service Area 3: Cleaner Production Assessments
Level Scale of Results Type of Evidence Strength of Evidence
Unknown Some Result Good Excellent Unavailable Leading Lagging Both Unavailable Weak Strong
Result Result Evidence Evidence
1. Outputs No 2 or fewer full 3-5 full 6 or more No evidence Systems in place Quantitative Leading No evidence Some data Extensive
information CPA’s/year or CPA’s/year full CPAs/yr available to conduct CP information and available available but data
available 5 or fewer or 6-15 rapid or 16 or assessments on number lagging not available,
rapid CPA’s/year more rapid of audited comprehensive covering all
CPA’s/year CPA’s/year companies CP
assessment
2. Outcomes No Less then 25 % Between 25 Over 75% of No evidence Systems in place Quantitative Leading No evidence Some data Extensive
information of CP options and 75% of CP options available to monitor follow information and available available but data
available have been CP options have been up on the on share of lagging not available,
implemented have been implemented implementation options comprehensive covering all
(or only implemented of implemented CP
qualitative recommendations assessments
information from CP
available on assessments
implementation
levels)
3. Impacts No Benefits Benefits Benefits No evidence Systems in place Quantitative Leading No evidence Some data Extensive
information achieved achieved achieved available to monitor information and available available but data
available quantified for quantified quantified environmental on benefits lagging not available,
less then 25% for 25 to for at least and productivity achieved by comprehensive covering all
of audited 75% of 75% of benefits achieved audited CP
companies audited audited after companies assessments
companies companies implementation
of
recommendations
from CP
assessment

88
Table 4.8: Findings from analysis of results for in-plant demonstrations (CP assessments)
Refer Table 4.7 for explanation of the categories
Service Area 3: CP Assessment
Level 1: Outputs Level 2: Outcomes Level 3: Impacts
Country Region Scale Evidence Type Evidence Strength Scale Evidence Type Evidence Strength Scale Evidence Type Evidence Strength
China Asia Excellent Both Weak Some Lagging Weak Some Lagging Weak
Columbia S America Excellent Both Strong Good Lagging Weak Unknown Unavailable Unavailable
Costa Rica C America Excellent Both Strong Good Lagging Weak Some Lagging Weak
Croatia CE Europe Some Both Weak Some Unavailable Unavailable Unknown Unavailable Unavailable
Egypt Africa Some Both Weak Good Both Weak Some Unavailable Unavailable
El Salvador C America Excellent Both Strong Good Both Strong Excellent Both Weak
Guatemala C America Excellent Both Strong Good Both Strong Excellent Both Strong
India Asia Good Both Weak Some Lagging Weak Unknown Unavailable Unavailable
Kenya Africa Excellent Both Weak Some Lagging Weak Some Lagging Weak
Mexico C America Excellent Both Strong Some Lagging Weak Some Lagging Weak
Morocco Africa Excellent Both Strong Good Lagging Weak Good Lagging Weak
Mozambique Africa Some Both Strong Some Lagging Weak Unknown Unavailable Unavailable
Nicaragua C America Excellent Both Strong Good Both Weak Good Both Weak
Peru S America Excellent Both Strong Good Lagging Weak Excellent Lagging Weak
South Africa Africa Some Both Strong Some Lagging Weak Unknown Unavailable Unavailable
Sri Lanka Asia Excellent Both Strong Good Lagging Weak Unknown Unavailable Unavailable
Uzbekistan Asia Excellent Both Strong Some Both Weak Some Unavailable Unavailable
Vietnam Asia Excellent Both Strong Good Lagging Strong Good Lagging Weak
Note: evidence categories are: leading = leading evidence only, lagging = lagging evidence only, and both = leading and lagging evidence.
89

89
The country-specific evaluation reports provide a detailed summary and review of the CP
assessment activities in the visited countries. For this summary a comparative analysis
was performed of the scope and results of these demonstration activities in the visited
countries. Semi-quantitative scales were therefore used, as in Table 4.7. The scales are
based on the number of assessment projects and the implementation status in assessed
companies. With this global programme-level summary it was not possible to properly
capture the quality and impact of these CP assessments more widely on the sectors and
clusters that the assessed companies are part of. The results of the classifications of the
visited countries are presented in Table 4.8, and Figure 4.3 presents the main analysis
results graphically.

Figure 4.3: Comparative analysis of results in regards to demonstrations/CP Assessments

CP Assessments

100% 0 0

90% 4
6
80%
8
1
70%
countries (n=18)

60%

50% 6

40%
13
30%
10
3
20%

10% 3

0% 0
Level 1: Outputs Level 2: Outcomes Level 3: Impacts

excellent good some unknown

Figure 4.3 shows that all visited NCPCs are active in delivering CP assessment services,
and the vast majority of them achieve ‘excellent’ (13 countries) or ‘good’ outcomes (1
country), as per the category descriptions in Table 4.7. The results at outcome level are
also good as the NCPCs could confirm that for all assessments undertaken at least ‘some’
implementation had followed by the companies, and in 10 countries it could be confirmed
that between 25 and 75% of the recommended CP options had been implemented (as
reflected in a ‘good’ rating on outcomes). The results at impact level are less clear, with
six countries each in the categories of ‘unknown’ and ‘some’ impact. Substantive impact
data are only available for 6 countries, rated as ‘excellent’ (3 countries which claim to
have investment and benefit data for over 75% of the audited companies) and as ‘good’ (3
countries have investment and benefit data for at least 25% of the audited companies).

Table 4.8 provides more detail in regard to datasets available at the NCPCs to
demonstrate results from CP assessments. All countries have leading and lagging
evidence to prove outputs, and in most countries the data are comprehensive (as reflected
in rating of the evidence strength as ‘strong’ for 13 countries). At outcome level, the
evidence base is weaker, as only 5 countries have both leading and lagging evidence, and

90
12 have only lagging evidence, resulting in the evidence basis being rated ‘weak’ in 14
countries. This implies that most NCPCs have data on the implementation status of some
but not necessarily the majority of CP options. At the level of impacts, the evidence base
is smallest, as NCPCs in only 10 countries maintain some kind of data on costs and
benefits of options implemented in the audited companies, most of these however only
lagging (7 countries) and incomplete (hence 9 countries with evidence rated as ‘weak’).

From the comparative analysis of results presented in Table 4.8 and Figure 4.3 it can be
concluded that CP assessments/in-plant demonstrations are indeed a core activity of the
NCPCs. Record keeping for the number of assessments undertaken (output level) is good
and shows sustained CP assessment activity over time. The follow up to CP assessments,
including monitoring of investments made and benefits achieved, have historically not
been strong, but it is now being recognised by most NCPCs as important, leading to some
kind of tracking, albeit not yet comprehensively, of the implementation of CP options in
audited companies in 17 countries. It should be noted, however, that a general trade-off
remains, as companies appear to be unwilling to pay for follow up and monitoring, while
NCPCs are encouraged to deliver services on a commercial basis.

4.4.4 Policy Advice

NCPCs engage with government and other stakeholders, including for example the
business community, academia and schools, to foster the development and adoption of
policy change conducive to the uptake of CP. While some NCPCs have been actively
advocating policy change right from their establishment, most have only done so after
having gained some national recognition through completion of CP demonstration
projects or otherwise. The potential to be active on policy matters is also influenced by
the host institutions. For example, those NCPCs hosted in industry associations or alike
tend to be primarily involved at executive level with for example support for national
implementation of MEAs (e.g. Morocco, Kenya, Colombia), promotion and
administration of voluntary agreements and/or development of sector guidelines and
standards (e.g. Guatemala). The NCPCs hosted in academia have been able to engage
with a broader set of government portfolios to advocate CP-conducive policy change (e.g.
Vietnam). For other NCPCs their national mandate to work on policy development has
been tightly limited (e.g. South Africa, Egypt). Finally, there are also several NCPCs that
conduct substantive policy relevant background studies that support the implementation
of CP-conducive policy, for example on harmonisation of environmental legislation
(Croatia), technical potential for CP (e.g. India and China) etc.

The achievements of the visited NCPCs on policy development are thus very diverse.
Specific remarks and suggestions have been provided on a country-by-country basis in
the respective country evaluation reports. An attempt is however made here to provide a
summary impression of the activity and results of all NCPCs. As with the other service
areas, a scaled system was developed to classify each NCPC in regard to the scope of its
policy advisory services, and the impacts thereof. This classification scheme is provided
in Table 4.9. The resulting classification of the visited NCPCs is provided in Table 4.10.
Figure 4.4 provides a graphical presentation of the overall results in policy advice.

91
 92
Table 4.9: Categories used in comparative analysis of NCPC results on CP policy advice in the visited countries
Service Area 4: Cleaner Production Policy Advice
Level Scale of Results Type of Evidence Strength of Evidence
Inactive Some Good Result Excellent Unavailable Leading Lagging Both Unavailable Weak Strong
Result Result Evidence Evidence
1. Outputs No Regular Regular Coordinated No evidence Systems in place Quantitative Leading No evidence Some data Extensive
specific liaison on submissions approach to available to record information on and available available but data
activity policy with policy draft CP- interactions with frequency and lagging not available,
from issues with suggestions to conducive government, types of comprehensive covering
NCPC government government policy and their content and government all NCPC
agencies lobby follow up interactions activity
government for
endorsement
2. Outcomes No Regular Regular Outsourcing of No evidence Systems in place Quantitative Leading No evidence Some data Extensive
specific invitations invitations policy available to record information on and available available but data
activity from from preparation communications frequency and lagging not available,
from government government to and/or from types of comprehensive covering
NCPC to NCPC to be part of implementation government, government all NCPC
comment on policy working tasks from their content and invitations activity
policy groups government to follow up
issues NCPC
3. Impacts No Recognition Inclusion of CP Enactment of No evidence Systems in place Qualitative Leading No evidence Some data Extensive
specific from in new CP- available to review information on and available available but data
activity government implementation conducive changes in scope and lagging not available,
from for CP of existing policies and government nature of comprehensive covering
NCPC policy policies and strategies policy and changes in all NCPC
advice strategies strategy policies and activity
received strategies and
their
implementation
arrangements

92
Table 4.10: Findings from analysis of results in CP policy advice
Refer Table 4.9 for explanation of the categories
Service Area 4: CP Policy
Level 1: Outputs Level 2: Outcomes Level 3: Impacts
Country Region Scale Evidence Type Evidence Strength Scale Evidence Type Evidence Strength Scale Evidence Type Evidence Strength
China Asia Excellent Both Strong Excellent Both Strong Excellent Both Strong
Columbia S America Good Lagging Weak Good Lagging Weak Some Lagging Weak
Costa Rica C America Excellent Both Strong Good Lagging Weak Good Both Weak
Croatia CE Europe Some Lagging Weak Good Lagging Strong Some Lagging Weak
Egypt Africa Inactive Unavailable Unavailable Inactive Unavailable Unavailable Inactive Unavailable Unavailable
El Salvador C America Excellent Both Strong Excellent Both Weak Good Both Weak
Guatemala C America Good Lagging Strong Good Lagging Strong Some Lagging Weak
India Asia Some Lagging Weak Good Lagging Strong Good Lagging Weak
Kenya Africa Good Both Strong Good Lagging Strong Excellent Lagging Strong
Mexico C America Some Lagging Weak Inactive Lagging Weak Inactive Lagging Weak
Morocco Africa Good Both Strong Good Lagging Strong Good Lagging Weak
Mozambique Africa Inactive Unavailable Unavailable Inactive Unavailable Unavailable Inactive Unavailable Unavailable
Nicaragua C America Excellent Both Strong Good Both Strong Good Lagging Weak
Peru S America Good Lagging Weak Some Lagging Weak Good Lagging Strong
South Africa Africa Some Lagging Strong Some Lagging Weak Some Lagging Weak
Sri Lanka Asia Good Both Strong Excellent Both Strong Excellent Lagging Strong
Uzbekistan Asia Inactive Unavailable Unavailable Inactive Unavailable Unavailable Good Unavailable Unavailable
Vietnam Asia Excellent Both Strong Excellent Both Strong Excellent Both Strong
Note: evidence categories are: leading = leading evidence only, lagging = lagging evidence only, and both = leading and lagging evidence.
93

93
Figure 4.4: Comparative analysis of results for policy advice
Policy Advice

100%

3 3
90% 4

80%
4 2 4
70%
countries (n=18)

60%

50%
6 8
7
40%

30%

20%
5
10% 4 4

0%
Level 1: Outputs Level 2: Outcomes Level 3: Impacts

excellent good some inactive

Figure 4.4 shows consistently high results at all three result levels, as per the
classification scheme provided in Table 4.9. 11 countries have ‘good’ or ‘excellent’
classification on policy output (i.e. NCPC is on regular basis in liaison with government
and provides recommendations on CP-conducive policy), 12 countries have a ‘good’ or
‘excellent’ classification on outcomes (i.e. NCPC is regularly invited to contribute to
policy formulation and/or implementation) and 11 countries have a ‘good’ or ‘excellent’
classification on impacts (i.e. enactment of CP-conducive policies and/or strategies),.
Some caution is needed for attribution, as in particular impacts through enactment of new
strategies and legislation, is not exclusively the result of activities of the NCPC. This
explains why for some companies the classification is lower for outputs, then for
subsequent outcomes and impacts (for example India, where new energy efficiency
legislation was enacted that fosters CP consideration and implementation (impact), and
the NCPC is involved in preparing technical/operational guidelines (outcome), but
appeared to have been only a minor party for creating the political commitment to
establish this legislation). For other countries, the reverse is true, i.e. that regardless of
significant effort from the NCPC to lobby for policy change, there has been hardly any
outcome or impact, apparently due to lacking commitment from other key stakeholders
(e.g. Costa Rica, Nicaragua and Peru).

Table 4.8 contains details on the type and strength of the evidence. It is remarkable that
lagging evidence dominates, as at least half of the countries for which some evidence was
available this was only lagging evidence, and this was equally so at the level of outputs,
outcomes and impacts. So, there is more information available about what was delivered
or achieved, rather than on systems or plans in place for engaging strategically and
tactically with government and other stakeholders on opportunities for effective CP
policies. This suggests that there is scope for better strategising in the policy activities of
the NCPCs.

94
From the comparative analysis of results on policy advice presented in Table 4.10 and
Figure 4.4.it can be concluded that 2/3rd of the visited NCPCs have achieved and/or
contributed to demonstrable CP-conducive policy change. Recordkeeping for intervention
in and contribution to policy processes is unfortunately weak, which add to the inherent
complexities of attribution of policy change to project activities. NCPCs may need to
develop a more strategic approach to policy change to increase results from policy advice
and have a reference for monitoring policy related activities.

4.4.5 Technology Transfer

Technology Transfer, specifically for Environmentally Sound Technologies (ESTs), was

added as an explicit aim and service area for the UNIDO-UNEP CP Programme from
1998. There are quite different expectations what is covered by technology transfer. It is
in some cases narrowly viewed as import of best available environmental process
equipment from donor or other industrialised, or, as the case might be, developing,
country to the NCPC host country. In a broader interpretation it entails all activities that
improve the demand and/or supply of environmental process technologies and know-how,
both locally as well as internationally (covering both North–South and South-South
transfers). In the narrow view, the programme’s success has been very limited as only for
a couple of countries specific international environmental technology transfers could be
identified that had come about as a result of the UNIDO-UNEP CP Programme directly
(e.g. Morocco, in olive processing industry) or indirectly (e.g. India, for the dyestuff
intermediates manufacturing industry). In the broader interpretation, quite a number of
NCPCs have made considerable progress in EST transfer, adaptation and replication,
through various initiatives that they themselves may not even regard as relevant for
technology transfer. These include for example: advisory services for establishment or
implementation of green credit lines (e.g. Vietnam and Columbia), definition of CP
standards for specific industry sectors (in particular in China), engineering drawings (or
‘blueprints’) for minor technology upgrades (in particular in India).

The activities relevant for EST transfer have been summarised for each of the visited
countries in the respective country review reports, and results analysed as a basis for
specific suggestions for future activities. Given the diversity of NCPC activities that are
supportive of EST transfer in the visited countries, classification of the respective
countries’ results exclusively on the volume or scope of EST transfer services was not
meaningful. For classification purposes a distinction was made between ‘bottom-up’ and
‘top-down’ approaches to technology transfer. ‘Bottom-up’ Approaches start with
technology needs assessment at company level, followed by technological capability
building, gap identification, technology sourcing and investment appraisal to initiate the
purchase, installation and adaptation of specific pieces of imported hardware. This
‘bottom up’ approach is most illustrative for the above-referred narrow interpretation of
technology transfer. ‘Top- down’ approaches are more commonly government-driven and
start with benchmarking and definition of environmental best practice standards that
companies will have to meet, forcing them to consider and adopt ESTs. This ‘top-down’
approach is more illustrative for the above-referred broad interpretation of technology
transfer. There is merit in combining both approaches, which has been classified as a
‘comprehensive approach’. Table 4.11 contains the details of the classification scheme
used for technology transfer results. The findings for the visited countries are summarised
in Table 4.12. Figure 4.5 provides a graphical presentation of the main findings.

95
96
Table 4.11: Categories used in comparative analysis of NCPC results on EST Transfer in the visited countries
Service Area 5: EST Transfer
Level Scope of Results Type of Evidence Strength of Evidence
Inactive Bottom-Up Top Down Balanced/ Unavailable Leading Lagging Both Unavailable Weak Strong
Comprehensive Evidence Evidence
1. Outputs No NCPC routinely NCPC NCPC No evidence Evidence for Records of Leading No evidence Some data Extensive
specific delivers routinely implements available accumulation nature and and available available but data
activity services for drafts coordinated of volume of lagging not available,
from technology environmental strategy to grow technological service comprehensive covering
NCPC needs and gap best practice demand and expertise, delivery all NCPC
assessment specifications supply for ESTs information specifically activity
for standard and tools in related to
setting by NCPC EST transfer
government
2. Outcomes No Regular Regular Outsourcing of No evidence Systems in Quantitative Leading No evidence Some data Extensive
specific requests to requests from policy available place to information and available available but data
activity NCPC for EST government to development to record on volume lagging not available,
from technology NCPC to NCPC, and some requests for and nature of comprehensive covering
NCPC assessment advice on EST specific success EST services EST service all NCPC
and/or standards for from top down and their requests activity
investment specific and/or bottom up follow up
advice sectors approaches
3. Impacts No Successful Adoption of EST-conducive No evidence Systems in Quantitative Leading No evidence Some data Extensive
specific implementation sector EST policy and available place to track information and available available but data
activity of EST in standards by strategy is being EST on number of lagging not available,
from specific government enacted, and investments ESTs comprehensive covering
NCPC companies some specific and policy transferred all NCPC
success from developments and further activity
top-down and/or disseminated
bottom up
approaches

96
Table 4.12: Findings from analysis of results on transfer of ESTs
Refer Table 4.11 for explanation of the categories
Service Area 5: EST Transfer
Level 1: Outputs Level 2: Outcomes Level 3: Impacts
Evidence
Country Region Approach Evidence Type Evidence Strength Approach Evidence Type Evidence Strength Scope Evidence Type Strength
China Asia Top-down Lagging Strong Top-down Both Strong Top-down Both Strong
Columbia S America Top-down Both Strong Top-down Both Strong Top-down Lagging Weak
Costa Rica C America Comprehensive Both Weak Bottom-up Both Weak Bottom-up Lagging Weak
Croatia CE Europe Inactive Unavailable Unavailable Inactive Unavailable Unavailable Inactive Unavailable Unavailable
Egypt Africa bottom-up Lagging Weak Inactive Unavailable Unavailable Inactive Unavailable Unavailable
El Salvador C America Comprehen-sive Both Weak Comprehen-sive Both Weak Bottom-up Lagging Weak
Guatemala C America Bottom-up Both Weak Bottom-up Both Weak Bottom-up Both Weak
India Asia Bottom-up Both Strong Top-down Lagging Strong Top-down Lagging Weak
Kenya Africa Bottom-up Lagging Weak Inactive Unavailable Unavailable Inactive Unavailable Unavailable
Mexico C America Bottom-up Leading Weak Inactive Leading Weak Inactive Leading Weak
Morocco Africa Comprehen-sive Both Strong Comprehen-sive Both Strong Comprehen-sive Both Weak
Mozambique Africa Inactive Unavailable Unavailable Inactive Unavailable Unavailable Inactive Unavailable Unavailable
Nicaragua C America Bottom-up Both Strong Comprehen-sive Both Strong Bottom-up Both Strong
Peru S America Inactive Unavailable Unavailable Inactive Unavailable Unavailable Inactive Unavailable Unavailable
South Africa Africa Inactive Unavailable Unavailable Inactive Unavailable Unavailable Inactive Unavailable Unavailable
Sri Lanka Asia Bottom-up Lagging Weak Inactive Unavailable Unavailable Inactive Unavailable Unavailable
Uzbekistan Asia Bottom-up Unavailable Unavailable Inactive Unavailable Unavailable Inactive Unavailable Unavailable
Vietnam Asia Comprehen-sive Both Strong Comprehen-sive Both Strong Comprehen-sive Both Strong
Note: evidence categories are: leading = leading evidence only, lagging = lagging evidence only, and both = leading and lagging evidence.
97

97
Figure 4.5: Comparative analysis of results for technology transfer (ESTs)
Technology Transfer

100%

90% 4

80%
9 9
70%
countries (n=18)

60%
8
50%
2
40% 4
2
30%
2
2
20%
5
10% 4
3

0%
Level 1: Outputs Level 2: Outcomes Level 3: Impacts

comprehensive top-down bottom-up inactive

Figure 4.5 shows that in 14 countries the NCPC is undertaking some activities that are
relevant for technology transfer, most of these through ‘bottom-up’ approaches (12
countries, respectively the total of countries classified as ‘bottom up’ and as
‘comprehensive’) emanating from extension of CP assessment services, followed by ‘top-
down’ approaches (6 countries, respectively the total of countries ranked as
‘comprehensive’ and ranked as ‘top down’). In half of the countries, the NCPC is ranked
inactive on technology transfer outcomes and impacts (9 countries). As discussed in
paragraph 4.4.4 on policy change, there could be a disconnection that results at outcome
and impact level are broader than at output level. A case in point is India, which at output
level has been most successful with ‘bottom up’ approaches for technology upgrades in
small scale industries using local manufacturing capability. At outcome and impact level,
India is rated as ‘comprehensive’ as the NCPC has been called in to undertake for the
Government of India technology studies to define best practice water and energy saving
technologies and practices for different sectors, and these have been incorporated into
government policy.

The type and strength of the evidence is also contained in Table 4.12. The NCPCs that
have been rated as being active in technology transfer generally have both leading and
lagging evidence available to underpin it (respectively for 8 countries on outputs and
outcomes and for 5 countries on impacts). However the evidence is in most cases
relatively weak as data on activities and results are not maintained on a routine basis.

The comparative analysis of results for technology transfer presented in Table 4.12 and
Figure 4.5 shows that in half of the visited countries the NCPC makes successful
contributions to EST transfer. The contribution is in many instances indirect, by
contributing to creating an enabling environment (e.g. with standard setting and
benchmarking) for EST investment. However on a case-by-case basis some NCPCs also
undertake technology gap assessment, technology sourcing and technology assessment
for selected companies and/or industry sectors.

98
4.5 National Assessments

The country reviews concluded with an evaluation of the activities and achievements at
the national level against the evaluation criteria set for this global programme evaluation.
As summarised in section 1.3 these were: relevance, effectiveness, efficiency and
sustainability, as the primary evaluation criteria, and ownership and capacity building, as
the secondary evaluation criteria. The findings from these 18 national evaluations are
covered in this section, for each of the evaluation criteria separately (paragraphs 4.5.1 to
4.5.6) and an integrative summary.

To enable transparent assessment scorecards were developed to capture elements that

would contribute to each of the main evaluation criteria. The evaluators completed these
scorecards, leading to national level assessments using a three-point ordinal scale,
respectively, ‘high’, ‘medium’ and ‘low’. For the summary at programme level this turned
out to mask all differences, and hence it was decided to expand to a five point ordinal
scale, respectively ‘excellent’, ‘good’, ‘satisfactory’, ‘poor’ and ‘absent’. This was done
in a manner that utilised the full performance range (26).

4.5.1 Relevance

Relevance is the first of the primary evaluation criteria. It results from a combination of
applicability (evidence or at least a reasonable expectation that the intended beneficiaries
have the financial, human, technical, managerial and other resources that are required to
implement CP) and value (evidence of at least a reasonable expectation that the intended
beneficiaries can gain a net benefit (financial, health and safety, environment, reputation,
etc) for themselves of their organisation from the implementation of CP).

A scorecard was developed and applied to assess the relevance, in regard to five
programme elements (respectively: CP concept, CP services, NCPC institution, regional
and global networking and technical assistance inputs) for three main target beneficiary
groups in the host country (respectively: private sector, government and
academia/research institutes). The scorecard with the basic results for the 18 visited
countries is provided in Table 4.13. Figure 4.6 and 4.7 provide the frequency.

Table 4.13: National assessment results for relevance (number of countries, total 18 countries)
Programme Ranking Beneficiaries (host country)
Elements Private Sector Government Academia
1. CP Concept Low 5 1 1
Medium 5 3 7
High 8 14 10
2. CP Services Low 2 1 10
(national) Medium 8 8 4
High 8 9 4
3. NCPC Low 5 1 5

26
This was achieved numerically, as per the following procedure. The low, medium and high values in the ordinal scale
were assigned numbers 1, 2 and 3 respectively, and were needed an un-weighted average of scores (S[old]) was calculated,
resulting in a number in the range between 1 and 3. This old score was then expanded to the 1-5 range into a new Score
(S[new], using the formula S[new] = 1+2*(S[old]-1). The S[new] was then rounded to the nearest integer, resulting in a
number 1, 2, 3, 4 or 5 which was then assigned to the new categories, respectively absent, poor, satisfactory, good or
excellent.

99
 Institution Medium 7 5 8
High 6 12 5
4. Networking Low 14 8 8
(regional and Medium 2 9 9
global) High 2 1 1
5. Technical Low 7 5 5
Assistance Medium 4 11 11
(international High 7 2 2
)

Figure 4.6: Relevance by beneficiary group (18 countries)

Relevance

7
7

6 6 6 6
6

5 5
5
countries (n=18)

4 4 4
4

3 3 3 3
3

2 2 2
2

1
1

0 0
0
Private sector Government Academia All beneficiaries

absent poor satisfactory good excellent

distributions for the aggregated results, respectively by beneficiary group and by

programme element. Cross reference to the respective countries can be found in the
summary table for all evaluation criteria, Table 4.19 in paragraph 4.5.7.

Figure 4.6 shows that overall relevance is rated relatively good in the majority of the
visited countries, respectively ‘excellent’ in 2 countries, ‘good’ in 5 countries,
‘satisfactory’ in 5 countries, and ‘poor’ in six countries, as in the set of columns of the
right (cross reference to the respective countries can be found in the summary table, Table
4.18 in paragraph 4.5.7). Among the three main beneficiaries, relevance rates highest for
government (rated ‘good’ or ‘excellent’ in total of 10 countries), followed by private
sector (rated ‘good’ or ‘excellent’ in total of 6 countries), followed by academia (rated
‘good’ or ‘excellent’ in 5 countries).

The evaluation of relevance was a result of various factors, in particular:

¾The alignment of CP with obligations under various MEAs to which the host
countries are signatory. This was present in all visited countries, and most strongly
for government;

100
¾Competition for national industries on domestic markets is on the rise, as are
customer demands, including for environmental performance, from overseas buyers,
as a result of trade liberalisation and globalisation of the national economies. This is
particularly prominent for the private sector and national government. However it is
not universally present in all visited countries; and

¾The worsening environmental burden caused by the manufacturing sector. Even

though this is evident in most countries, industry’s environmental impact is still only
regarded a national priority in some of the visited countries (e.g. China, India,
Vietnam, South Africa, Egypt, Mexico).

There is a trend in all countries that each of these factors gains importance, supporting the
expectation that CP will be increasingly relevant in the future. However, the relative rate
of change in these driving factors for CP relevance is also quite different among the
visited countries.

Figure 4.7: Relevance by programme element (18 countries)

Relevance
10
10
9
9

8
7 7
7
6 6 6
countries (n=18)

6
5 5 5 5
5
4 4 4 4
4
3 3
3
2 2 2 2 2 2
2
1 1 1
1
0 0 0 0
0
CP concept CP services NCPC institution Networking TA inputs overall

absent poor satisfactory good excellent

Figure 4.7 reveals quite large differences in relevance between the five key programme
elements. Overall the relevance of the national components is rated high, with at least half
of the countries achieving a ‘good’ or ‘excellent’ relevance score for CP concept (13
countries), NCPC institution (10 countries) and CP services (9 countries). In half of the
countries the relevance of the CP services is rated ‘poor’. This is partially a reflection of
the fact that only a few NCPCs (e.g. Sri Lanka, China, Mexico) have developed services
that are particularly catered to academia (leading to lack of relevance for one beneficiary
group pulling down the average score). A compounding factor is however that the
standard CP services are catered to the manufacturing sector, and in countries with
limited development of this sector, opportunities to develop the CP concept specifically to
sectors of national priority has not sufficiently taken place (e.g. agriculture, forestry,
fisheries, crafts sectors, as for example in Mozambique, Sri Lanka and Kenya).

101
As per Figure 4.7 the relevance of the international components is rated markedly lower,
as reflected by the fact that the relevance of international expert inputs and of networking
is rated ‘excellent’ or ‘good’ in only 7 (international technical assistance) or 2 (network)
of the visited countries. This appears in part a reflection of the current low intensity of
networking (for nearly all countries) and technical assistance inputs (in many of the
visited countries no further technical assistance inputs are being provided as the
institutional funding period has ended, or as only a very limited budget had been
allocated). Strictly speaking, even in those countries there could still be an expectation
that more intensive networking and more substantive technical assistance could be
beneficial, even through the NCPC currently manages to operate without such.

4.5.2 Effectiveness

Effectiveness is the second of the primary evaluation criteria. It addresses whether or not
the combination of the national centres, their networking and management and the
technical assistance they receive, enable the uptake of CP practices, technologies and
policies by the intended beneficiaries in the host countries.

A scorecard was developed and applied for each of the visited countries. The results are
presented in Table 4.14. This table reveals that the effectiveness of the programme to
establish NCPCs for CP service delivery has been generally good. This is further
illustrated with Figure 4.8 which shows that integrated across the programme components
the effectiveness was rated as ‘good’ or ‘excellent’ in half of the visited countries
(respectively in 5 and 4 countries) and ‘satisfactory’ in one third (6 countries). Cross-
reference to the respective countries can be found in the summary table, Table 4.19 in
paragraph 4.5.7.

Table 4.14: National assessment results for effectiveness (18 countries)

Programme Effectiveness Score
Component Contributing Elements Intended Result Rating No of
Countries
1. Programme Programme Strategy; Assist Centre and host Low 6
Management Liaison with Programme organisation with the Medium 9
Stakeholders and establishment and High 3
Donors; Planning and operation of an NCPC
Reporting; Budget and
Financial Control;
Mentoring and Coaching
2. National Information Uptake of CP by Low 1
Centre Dissemination; Training; companies Medium 9
In-plant Demonstrations; CP awareness High 8
Policy Advice; EST CP-conducive policy
Transfer change
3. Technical Specialist Expertise/ Improve the capability Low 2
Assistance Consultant; Training of of the NCPC to deliver Medium 10
NCPC Staff; Resource effective CP services in High 6
Materials; CP Award professional manner
Scheme
4. Networking (Annual) Directors’ Assist NCPC to utilise Low 4
Meeting; Regional complementary skills Medium 9
Cooperation; Publication and know-how from High 5
and Promotion ‘sister’ NCPCs

102
Among the programme components, the effectiveness was rated highest for the national
centre (as per Table 4.14, rated ‘high’ or ‘medium’ in 17 countries), followed by technical
assistance (rated ‘high’ or ‘medium’ in 16 countries) and networking (rated ‘high’ or
‘medium’ in 14 countries). The effectiveness was ranked lowest for programme
management (rated ‘low’ in 6 countries). This reflects the fact that the NCPCs that have
operated for a number of years without institutional funding through the UNIDO-UNEP
CP Programme, displayed differing degrees of alienation from the Programme (e.g.
China, India, Mexico, Croatia). These centres are barely aware of changes in direction in
the Programme (e.g. in regard to introduction of new service areas) and are at best in
irregular contact with the programme management unit. Even though it can be argued that
for these NCPCs some elements of the programme management do no longer apply
(financial control, donor liaison etc.) by virtue of their ongoing association with the
UNIDO-UNEP CP Programme, it is portrayed that they are still to some extent influenced
by the overall direction of the Programme and should have be heard by the programme
management unit, in regard to for example planning of networking opportunities.

Figure 4.8: Effectiveness scores for visited countries (18 countries)

Effectiveness
6
6

5
5

4
4
countries (n=18)

3
3

0
0
integrated score

absent poor satisfactory good excellent

4.5.3 Efficiency

The third of the primary evaluation criteria is efficiency. It pertains to maximising the
results (outputs outcomes and impacts, as detailed in section 4.4) within the limits of the
resources available to the NCPC, including financial, human, technical and
organisational/institutional resources.

A scorecard was developed for rating the efficiency of the different programme
components, and applied for the 18 visited countries. Table 4.15 contains the summary of
the findings. This table reveals that the efficiency of the programme to establish NCPCs
for CP service delivery has been adequate. This is further illustrated with Figure 4.9

103
which shows that integrated across the programme components the efficiency was rated
as ‘excellent’ in 3 countries, ‘good’ in 4 countries and ‘satisfactory’ in another 6
countries. Cross-reference to the respective countries can be found in the summary table,
Table 4.19 in paragraph 4.5.7.

Table 4.15: National assessment results for efficiency (18 countries)

Programme Elements Efficiency Score
Component Implementation Arrangements Rating No of
Countries
1. Programme Centrally through UNIDO (agency Low 5
Management implementation) Medium 10
High 3
2. National Centre Created within existing host institution Low 2
Medium 5
High 11
3. Technical Provided through International Reference Low 3
Assistance Centres Medium 11
High 4
4. Networking Coordinated centrally by programme Low 8
management unit in UNIDO headquarters Medium 6
High 4

Figure 4.9: Efficiency scores for visited countries (18 countries)

Efficiency
6
6

5
5

4
4
countries (n=18)

3
3

0
0
integrated score

absent poor satisfactory good excellent

From Table 4.15 it can further be concluded that among the programme components, the
national centre scored best in regard to efficiency, as reflected in a rating as ‘high’ in over
60% of the visited countries (11 countries). The NCPCs are generally professionally
operated and managed and achieve a sustained level of outputs with in many cases fairly
modest budgets. However, further to the comments made throughout section 4.4 it should
be noted that efficiency could only be ascertained in regard to outputs (activities
undertaken) due to lack of information on outcomes and impacts. In several of the
countries, the evaluators found that a more targeted approach with fewer, but more

104
strategic outputs, would have potential to increase outcomes and impact from the
Programme. However in the current approach to measure outputs, this could have a
perceived negative impact on efficiency.

The relatively high number of countries achieving only a ‘low’ efficiency score on
programme management (5 countries) and on networking (8 countries) are reflective of
the issues discussed in section 4.3 on national implementation of the UNIDO-UNEP CP
Programme. In regard to programme management, this is a volume problem, as NCPCs
spent too much of their available time and resources on meeting the programme
requirements for project administration and financial control. Even though this applies to
all countries, there are considerable differences among countries in regard to the degree to
which UN staff involved locally and/or at headquarters succeeds in easing the
administrative and budgetary burdens for the respective NCPC. NCPCs have invested
much less efforts into networking then in meeting administrative requirements. However,
due to lack of follow up, or, as the case might be, perception thereof, from the Programme
management, there has been hardly any output or outcome for the NCPC from the effort it
put into networking, leading to a low efficiency rating at national level for networking.

4.5.4 Sustainability

Sustainability is the fourth and final of the primary evaluation criteria. It covers the
probability or likelihood that the benefits achieved from the UNIDO-UNEP CP
Programme will continue into the future, at a level equal to achievements during
programme implementation (‘continuing’), or at levels greater (‘expanding’) or smaller
(‘declining’) than during programme implementation. Such benefits include the
availability of CP services (or the outputs from the current programme), the productivity
gains and environmental benefits from CP uptake (or the outcomes from the current
programme), and the overall contribution of CP to sustainable industrial development of
the host country (or impact of the current programme). It is worth re-iterating here that
this interpretation of sustainability is different from the most frequently used
interpretation by the NCPCs, the programme management and the current donors, namely
as the financial independence of the respective NCPCs as institutions for CP service
delivery.

The sustainability has been estimated on the basis of the actual or likely presence of
drivers/incentives for CP, or more generally, programme sustainability factors, including:

ƒ Willingness of target industries, governments and/or other organisations (including

current and potentially other donors) to pay for the provision of CP services;

ƒ Continued availability of the know-how and skills to deliver high quality and
effective CP services;

ƒ Consensus about the relevance and benefits of CP (‘critical mass’);

ƒ Presence of framework conditions conducive to CP (e.g. legislative framework,

policy, tax, financial incentives, etc.);

ƒ Technology push (availability of new CP technologies and practices customised to

local industry needs and capabilities);

105
ƒ Market push for CP (through prices for water, energy, waste, materials, etc.); and

ƒ Market pull for CP (exerted through the supply chains that the target industries are
part of or would aspire to become part of).

A detailed scorecard was prepared as the basis for the assessment on sustainability of the
programme at the national levels. The results are presented in Table 4.16 and 4.10. Figure
4.10 presents the integrated assessment based on consideration of availability of CP
services, environmental and productivity benefits and catalyst role for sustainable
industrial development. The overall sustainability of current programme benefits is rated
‘excellent’ for 4 countries, ‘good’ for 6 countries, ‘adequate’ for another 6 countries and
‘poor’ for the remaining 2 countries. Cross-reference to the respective countries can be
found in the summary table, Table 4.19 in paragraph 4.5.7.

Table 4.16 shows the constituent ratings for the three categories of programme benefits.
This reveals that the scores on sustainability are dominated by the high scores on
sustainability of the environmental and productivity benefits achieved from CP uptake.
This was rated ‘high’ in the vast majority of countries (15). This reflects the high degree
of certainty that companies that have implemented CP options will continue to do so in
the future, as they will seek to maintain the real time benefits they are achieving from
doing so. There is some concern about waning off of the benefits from good
housekeeping and other softer low or no cost options, as people and organisations tend to
revert back to old habits. However there is a reasonable expectation that this waning off
will be compensated through gradual increases in the share of options implemented.
However, no autonomous step change in the level of benefits can be achieved.

The continued availability of CP services at a comparable level then during programme

implementation is also likely, and therefore rated ‘medium’ in 11 countries and ‘high’ in
another 6 countries. Even if the NCPC would dissipate, its staff would most likely
continue to practice its CP skills in a different set up. However, over time the currency
and quality of services is likely to decline, in the absence of continued professional
development and other opportunities to benchmark and improve skills. Albeit lowest
among the benefit categories considered, the sustainability of the catalyst function for
sustainable industrial development is also still reasonably good (rated as ‘high’

Table 4.16: National assessment results for sustainability (18 countries)

Programme Benefits Sustainability Score
Rating No of Countries
1. Availability of CP Services Low 2
Medium 11
High 5
2. Productivity and Environmental Benefits Low 0
Medium 3
High 15
3. Catalyst for Sustainable Industrial Development Low 7
Medium 6
High 5

106
Figure 4.10: Sustainability scores for visited countries (18 countries)
Sustainability

6 6
6

4
4
countries (n=18)

2
2

0
0
integrated score

absent poor satisfactory good excellent

for 5 countries and ‘medium’ for 6 countries). This is also largely attributed to the people
factor, in that skilled CP professionals will remain active as individual lobbyist for CP,
albeit of course less effective than done from within an institutional framework.

Overall however some care is needed with the interpretation of the sustainability scores,
as they related to the current level of CP uptake. As discussed in previous paragraphs and
section 4.4, the impact of the Programme is in most countries still relatively modest. Even
if benefits are maintained at this level, one cannot expect that CP dissemination and
implementation is from now on an autonomous process that will achieve widespread
uptake of CP in the near future without further support.

4.5.5 Capacity Development

Capacity development is the first of the secondary evaluation criteria. It refers to the
extent that the programme develops essential capacities for local stakeholders to improve
their current and future well being. It is related to the primary evaluation, in particular on
effectiveness and efficiency.

Four target capacities were distinguished, respectively:

1. Resource Productivity: the efficient utilisation of natural resources (materials, energy,

water, etc.) for the production of goods and services that bring quality of life;

2. Environmental Management: minimising the impact of business on the environment

to protect the health of workers and community and the ecological integrity of the
natural environment;

3. Entrepreneurship: skills, tools and systems of the owners/operators of businesses to

run their businesses in a rational and planned way achieving a solid balance between
short term profit and medium to long term viability; and

107
4. Public Private Partnership: recognition by government and business sector that
collaboration on issues of national concern (including environmental management
and productivity) is necessary and the skills to do so.

In addition three principal target groups were singled out for capacity development,
respectively individual enterprises (in particular those having received services directly or
indirectly from the NCPC), the private sector (industry peak bodies, sector associations
and professional associations) and government (national and sub-national level in
different portfolios).

A scorecard was prepared to assess the programme’s capacity development achievements

at the national level. The results are presented in Table 4.17. Figure 4.11 and 4.12 provide
the frequency distributions for the aggregated results, respectively by target group and by
target capacity.

Table 4.17: National assessment results for capacity development (number of countries, total 18
countries)
Target Capacities Ranking Target Groups (in host country)
Enterprises Private Government
Sector
1. Resource Low 1 6 4
Productivity Medium 4 10 11
High 13 2 3
2. Environmental Low 1 4 3
Management Medium 10 13 6
High 7 1 9
3. Entrepreneurship Low 12 17 15
Medium 6 1 3
High 0 0 0
4. Public Private Low 13 10 10
Partnership Medium 3 2 1
High 2 6 7

Figure 4.11 reveals on average among all target groups a reasonable degree of capacity
building. This is evidenced in the last set of bars, showing that in 8 countries capacity
building averaged over the three target groups was rated as ‘satisfactory’ and in 3
countries as ‘good’. Among the three target groups, capacity development was most
profound among individual enterprises, for which capacity development was evaluated as
‘good’ in 6 countries and ‘satisfactory’ in 10 countries. The results for the two other
target groups, private sector and government, are identical, namely ‘excellent’ in 1
country, ‘good’ in 1 country, ‘satisfactory’ in 5 countries, ‘poor’ in 8 countries and
‘absent’ in 3 countries. However, these are not necessarily the same sets of countries.
Cross-reference to the respective countries can be found in the summary table, Table 4.19
in paragraph 4.5.7.

108
Figure 4.11: Capacity development by target group (18 countries)
Capacity building
10
10

9
8 8 8
8

7
6
countries (n=18)

6
5 5 5
5

4
3 3 3
3
2 2
2
1 1 1 1
1
0 0 0
0
Enterprises Private Sector Government All Stakeholders

absent poor satisfactory good excellent

Figure 4.12: Capacity development by target capacity (18 countries)

Capacity building
9 9 9
9
8 8 8
8

7
6
6
countries (n=18)

5 5
5
4 4
4
3 3
3
2 2 2
2
1 1 1
1
0 0 0 0 0 0
0
Resource Productivity Environmental Entrepreneurship Public Private All Capacities
Management Partnership

absent poor satisfactory good excellent

Figure 4.12 provides further background on capacity development. It summarises results

by the target capacity. The average over all capacities (set of bars on the far right side) is
identical to the average for all stakeholders in Figure 4.11. Figure 4.12 however shows
that capacity development in two target capacities, respectively resource productivity and
environmental management, is good, and in the two other target capacities, respectively
entrepreneurship and public private partnerships, capacity development has been minimal
in the vast majority of the countries. The overall results are thus pulled down by the near
absence of capacity development in entrepreneurship and public-private-partnerships.

109
Despite their presence in programme documents (see Section 2.2), programme delivery in
the host countries is not geared towards delivery on those capacities.

Figure 4.12 demonstrates a slightly higher assessment on capacity building for resource
productivity (rated ‘excellent’ in 2 countries and ‘good’ in 9 countries) than for
environmental management (rated ‘excellent’ in 1 country and ‘good’ in 8 countries).
This difference is however very minor. There is however a tendency for many NCPCs to
either focus slightly more on resource productivity (including energy savings etc., as for
example in India) while other focus more on environmental management (reduction of
waste, waste water and air emissions, as for example in Sri Lanka).

4.5.6 Ownership

Ownership is the second of the secondary evaluation criteria. It reflects upon the
commitment of local stakeholders to maintain the CP programme, locally in the host
country, as well as globally through the UNIDO-UNEP CP Programme. It could cover
(co-)funding of centre activities, providing expert inputs, implementation of policy
conducive to CP implementation and other forms of recognition and endorsement).
Ownership is related to the primary evaluation criteria, in particular relevance and
sustainability.

In assessing ownership a distinction was made between ownership of the CP concept (as a
business practice and environmental improvement tool), of the national centre (as a CP
service delivery organisation) and of the global programme. The results are presented in
Table 4.18. Figure 4.13 and 4.14 provide the frequency distributions for the aggregated
results, respectively by stakeholder grouping and by programme element.

Table 4.18: National assessment results for ownership (number of countries, total 18 countries)
Target Capacities Ranking Stakeholders (in host country)
Enterprises Private Government
Sector
1. Ownership of CP (concept, business Low 7 4 1
practice, environmental Medium 8 9 3
improvement tool) High 3 5 14
2. Ownership of national centre Low 12 10 1
(institution for CP service delivery) Medium 4 3 9
High 2 5 8
3. Ownership of global programme Low 16 13 12
(UNIDO-UNEP CP network) Medium 2 3 6
High 0 2 0

Figure 4.13 reveals on average among all stakeholder groupings a fair level of ownership.
This is evidenced in the last set of bars (furthest to the right), showing that in 3 countries
capacity building averaged over the three stakeholder groupings was rated as
‘satisfactory’, in 4 countries as ‘good’ and in 1 country as ‘excellent’. Among the three
national stakeholder groupings, ownership was most profound among government, for
which ownership was evaluated as ‘good’ in 9 countries and ‘satisfactory’ in 5 countries.
Ownership between the two other stakeholder groupings is markedly lower. The private
sector (associations, peak industry bodies etc) however display a slightly higher level of
ownership than individual enterprises, as evidenced by total of countries evaluated as

110
‘excellent’ or ‘good’ being 5 for private sector and 2 for enterprises. Cross-reference to
the respective countries can be found in the summary table, Table 4.19 in paragraph 4.5.7.

Figure 4.13: Ownership by stakeholder grouping (18 countries)

Ownership
9 9
9

8
8

7 7
7

6
countries (n=18)

5
5

4 4 4
4

3 3 3
3

2 2
2

1 1
1

0 0 0 0
0
Enterprises Private Sector Government All Stakeholders

absent poor satisfactory good excellent

Figure 4.14 provides further background on ownership. It summarises the assessment

results by programme elements. The average over all programme elements (set of bars on
the far right side) is identical to the average for all stakeholders in Figure 4.13. Figure
4.14 however displays a great difference in the level of ownership between the
programme elements. Ownership over the CP concept (i.e. as business practice and
environmental improvement tool) is by far the highest, with half of the countries having
‘excellent’ (3 countries) or ‘good’ ownership (6 countries). Ownership of the national
centre is still modest, with 2 countries evaluated as ‘excellent’, and 3 countries each as
‘good’

111
Figure 4.15: Ownership by programme element (18 countries)
Ownership
12
12

10
9

8
countries (n=18)

6
6
5

4 4
4
3 3 3 3 3 3

2 2
2
1 1 1

0 0
0
CP Concept National Centre Global Programme All Elements

absent poor satisfactory good excellent

or ‘satisfactory’. The level of ownership of the global programme is very low, rated as
‘absent’ in 2/3rd of the countries. The later appears to reflect that the networking and
technical assistance inputs are not profoundly present in most countries.

4.5.7 Overall Assessment

The previous paragraphs discussed the national assessments against the programme
evaluation criteria (respectively: relevance, effectiveness, efficiency, sustainability,
capacity development and ownership). A comparative summary covering all criteria is
provided here.

Table 4.19 is provided as a cross-reference table. It provides for each of the 18 visited
countries the detailed assessment ratings. These are not further discussed here. These are
provided here to provide a link to the country evaluation reports.

112
Figure 4.15: Summary of results of national level evaluation on programme level evaluation
criteria
Summary of National Assessments

100% 0 0 0 0
2 2
3 3
90% 4
6
80%
5
6
70%
6
7
countries (n=18)

60% 6

5
50%

40% 6 8
5 3
30% 5
5
20%
4

10% 4 4
3 3
2
1
0% 0
Relevance Effectiveness Efficiency Sustainability Capacity Ownership
Development
Evaluation Criteria

excellent good satisfactory poor absent

Figure 4.15 shows the frequency distributions of all countries on all six evaluation
criteria. This figure illustrates that the distributions are quite similar for the four primary
evaluation criteria. The highest score among these four criteria is achieved for
sustainability (10 countries achieving either ‘excellent’ or ‘good’ assessment), closely
followed by effectiveness (9 countries in these two categories), efficiency (8 countries in
these two categories) and relevance (7 countries in these two categories). Each of these
thus achieved a score in either of the two highest categories for 39 to 56% of the visited
countries. In light of ongoing concerns about sustainability of NCPCs by the programme
management and donors, this is somewhat surprising. It is explained by the fact that this
programme evaluation took a different interpretation of sustainability, compared to the
prevailing interpretation of sustainability as financial independence of the NCPC from the
UNIDO-UNEP CP Programme. Environmental and productivity benefits from CP
implementation in businesses and CP trained staff that can deliver CP services will very
likely continue, at least at the current levels. The weakest score among the primary
criteria for relevance suggests that more can be done to tailor CP concepts and practices
to national priorities (in regards to key sectors of economy (e.g. rural and service sectors),
and socio-economic and environmental objectives).

Figure 4.15 also shows that the assessment on the secondary criteria is markedly weaker
than on the primary criteria. Focusing again on the two highest-ranking categories, these
are only achieved in 5 countries for ownership and 3 countries for capacity development.
Country level implementation of the UNIDO-UNEP CP Programme appears to be
focused towards delivery of short term environmental and productivity benefits, and this
appears to somewhat overshadow the potential for longer term benefit through capacity
development and co-ownership of the CP programme. It should however also be pointed
out that in both cases, this overall result is dragged down by an interpretation of

113
ownership and capacity development that is different from those commonly used within
the Programme. Even though these interpretations are supported by the Programme’s
documents (see section 2.2) they are not focused upon in programme delivery and
national implementation by the NCPC. In case of ownership, this involved extension of
ownership from just ownership of centre, to also include ownership of the CP concept
(which improved the overall assessment on ownership) and ownership of the global
programme (which reduced the overall assessment on ownership). This was further
enunciated by considering ownership separately for enterprises and the private sector,
compared to a narrower view considering only ownership from, or on behalf of, the
national government. In case of capacity development, this programme evaluation did
cast the net wider to include consideration for capacities in regard to entrepreneurship and
public-private partnerships. Both turned out to score very low, in turn lowering the
overall assessment on capacity development.

114
Table 4.19: Detailed summary table of the national assessments
Evaluation Criteria China Columbia Costa Rica Croatia Egypt El Salvador Guatemala India Kenya
1. Relevance Satisfactory Satisfactory Good Poor Poor Good Good Satisfactory Poor
Private sector Satisfactory Satisfactory Good Poor Poor Good Excellent Poor Poor
Government Excellent Satisfactory Good Satisfactory Satisfactory Good Good Good Good
Academia Poor Satisfactory Satisfactory Poor Absent Satisfactory Good Absent Absent
CP concept Good Good Excellent Poor Absent Excellent Excellent Good Good
CP services Good Good Good Poor Poor Good Good Poor Poor
NCPC institution Poor Good Good Satisfactory Satisfactory Good Excellent Poor Satisfactory
Networking Poor Poor Poor Poor Poor Poor Satisfactory Poor Poor
TA inputs Good Satisfactory Good Poor Satisfactory Good Good Satisfactory Poor
2. Effectiveness Poor Good Good Good Satisfactory Excellent Excellent Poor Satisfactory
3. Efficiency Satisfactory Good Good Satisfactory Satisfactory Excellent Good Poor Satisfactory
4. Sustainability Good Excellent Good Poor Satisfactory Excellent Good Satisfactory Satisfactory
5. Capacity Development Poor Satisfactory Good Absent Poor Good Satisfactory Poor Satisfactory
Enterprises Satisfactory Good Good Poor Satisfactory Good Good Satisfactory Satisfactory
Private Sector Poor Satisfactory Satisfactory Poor Poor Excellent Satisfactory Absent Poor
Government Satisfactory Satisfactory Good Absent Poor Good Satisfactory Poor Satisfactory
Resource Productivity Poor Good Good Poor Good Excellent Good Satisfactory Good
Environmental Management Good Satisfactory Good Poor Satisfactory Excellent Satisfactory Poor Good
Entrepreneurship Absent Poor Poor Absent Absent Poor Absent Absent Poor
Public Private Partnership Absent Good Good Absent Absent Good Excellent Absent Absent
6. Ownership Satisfactory Satisfactory Good Absent Poor Good Good Poor Poor
Enterprises Poor Poor Poor Absent Absent Satisfactory Poor Poor Absent
Private Sector Poor Poor Good Absent Poor Excellent Good Poor Poor
Government Good Satisfactory Good Poor Good Good Good Satisfactory Poor
CP Concept Good Good Good Poor Satisfactory Excellent Good Good Poor
National Centre Satisfactory Satisfactory Good Poor Poor Good Good Poor Poor
Global Programme Absent Absent Poor Absent Absent Satisfactory Poor Absent Absent
115

115
 116
Table 4.19: Detailed summary table of the national assessments (continued)
Evaluation Criteria Mexico Morocco Mozambique Nicaragua Peru South Africa Sri Lanka Uzbekistan Vietnam
1. Relevance Good Excellent Poor Excellent Poor Satisfactory Satisfactory Poor Good
Private sector Satisfactory Excellent Absent Excellent Poor Satisfactory Poor Poor Good
Government Satisfactory Excellent Poor Excellent Poor Satisfactory Good Satisfactory Excellent
Academia Excellent Good Poor Excellent Poor Satisfactory Satisfactory Satisfactory Good
CP concept Excellent Excellent Poor Excellent Good Excellent Satisfactory Satisfactory Good
CP services Excellent Excellent Poor Excellent Poor Poor Poor Poor Excellent
NCPC institution Satisfactory Excellent Poor Excellent Absent Good Good Good Good
Networking Poor Good Absent Good Absent Absent Poor Absent Satisfactory
TA inputs Satisfactory Good Absent Excellent Absent Poor Poor Poor Good
2. Effectiveness Good Excellent Poor Excellent Satisfactory Satisfactory Satisfactory Satisfactory Good
3. Efficiency Good Excellent Poor Excellent Poor Satisfactory Poor Satisfactory Good
4. Sustainability Satisfactory Excellent Poor Excellent Good Good Satisfactory Satisfactory Good
5. Capacity Development Poor Satisfactory Absent Good Poor Satisfactory Satisfactory Satisfactory Satisfactory
Enterprises Satisfactory Good Poor Good Satisfactory Satisfactory Satisfactory Satisfactory Satisfactory
Private Sector Absent Satisfactory Absent Good Poor Satisfactory Poor Poor Poor
Government Absent Good Poor Good Poor Satisfactory Satisfactory Satisfactory Good
Resource Productivity Poor Good Absent Excellent Poor Satisfactory Good Good Good
Environmental Management Poor Good Poor Good Poor Good Good Satisfactory Good
Entrepreneurship Absent Poor Absent Poor Absent Absent Poor Poor Poor
Public Private Partnership Absent Good Absent Good Poor Good Absent Poor Poor
6. Ownership Absent Excellent Absent Good Poor Satisfactory Poor Poor Poor
Enterprises Absent Good Poor Good Absent Poor Absent Absent Poor
Private Sector Poor Excellent Absent Good Poor Poor Poor Absent Absent
Government Poor Good Poor Good Satisfactory Good Satisfactory Satisfactory Good
CP Concept Poor Excellent Poor Excellent Satisfactory Good Satisfactory Poor Satisfactory
National Centre Absent Excellent Poor Excellent Poor Satisfactory Poor Poor Poor
Global Programme Absent Good Absent Satisfactory Absent Absent Absent Absent Poor

116
 Part II:

Analysis & Assessment

117
5
Portfolio Analysis
_______________________________________

5.1 Introduction

The key findings from each of the three ‘pillars’ of this programme evaluation have been
covered in the previous chapters, respectively from the review of programme documents
(Chapter 2), of the self evaluations (Chapter 3) and of the independent country
evaluations (Chapter 4). This chapter analyses the findings from these three ‘pillars’ in an
integrated manner, with a view to analyse similarities and differences in the establishment
and operation of NCPCs/NCPPs. The analysis of the portfolio of activities and
institutional arrangements of the NCPCs/NCPPs is made to gain a better understanding of
the current richness and diversity in the UNIDO-UNEP CP Programme and identify
possible avenues to bolster this as the Programme evolves further. The detailed
programme-level assessment on the programme evaluation criteria is covered in the
companion Chapter 6 (programme assessment).

This chapter is thus analytical and not intended to be evaluative or judgemental. The
analysis is complemented with suggestions for further development of concept, methods,
tools and institutional arrangements for the Programme. These are presented here to
illustrate how the findings from the portfolio analysis can shed new light on the
Programme. Moreover in its recommendations (in Chapter 7) this evaluation refers back
to the typologies and terminology derived from the portfolio analysis presented here. In
so doing, this chapter provides the core ideas for the recommended changes in the
Programme.

The remainder of this chapter is organised in four sections. Section 5.2 provides a
background on key factors that have contributed to the current diversity among the
NCPCs/NCPPs. Section 5.3 then analyses differences at institutional level, followed by
an analysis for the main service areas (section 5.4). The final section (section 5.5)
discusses practical ways forward for managing the diversity of CP initiatives at the
national level to achieve success at the global programme level.

5.2 Background

The UNIDO-UNEP CP Programme has been designed and implemented to use a quasi-
standardised model approach for development of national entities for CP service delivery
that would undertake information dissemination, training, in-plant demonstrations, policy
advice and technology transfer. This evaluation confirms that after 13 years a diverse set
of national centres/programmes has evolved, each which a high degree of uniqueness.
The project model for the NCPC evolved in each of the countries, influenced by a variety
of factors. In analysing the roots of the current diversities at the national level, it is
worthwhile to differentiate between internal factors (those controlled or at least to a

119
considerable degree controllable by the CP Programme) and external factors (those that
are not under direct control of the CP Programme, but that the Programme can adapt to).
Figure 5.1 provides a schematic presentation for such roots of diversification. These
categories are provided here to understand differences, so that these can be considered in
a meaningful way in the remainder of this chapter for analysing the different institutional
arrangements and operational models. Also, the categories of internal and external factors
may not necessarily cover all relevant factors.

The internal factors can be clustered at three levels (or scales), respectively centre, project
and programme level.

At centre level, diversification is created by the host institution (its own mission and
mandate (e.g. technical institute, university or industry association), its reputation
with key stakeholders in the public and private sectors, its own in house technical,
managerial and analytical capabilities, etc.), the centre’s governance structure
(accountability and transparency, stakeholder involvement in oversight of the host
institution etc.) and director and other key centre staff (their disciplinary background,
professional experience and standing, management and networking skills and other
personal attributes).

Figure 5.1: Roots of diversification in CP Programme

Programme Factors
Programme strategy
Administrative and reporting
requirements Status of Knowledge
Special projects Past CP experience
National system of innovation
Project Factors
Donor requirements
Host government
Detailed project design
State of Environment
State of Economy
Centre Factors Natural resources
Size, structure of economy
Host institution Pollution levels
Framework conditions
Governance Environmental policy and
Development priorities
Key staff management

Internal External
Factors Factors

ƒ At the project level, diversification is being created by project level features, e.g.
donor government requirements and commitments, project design and funding levels,
partner agency in the host government (e.g. environment, trade or science ministry)
and nature, quality and volume of international expert inputs, training, networking
and knowledge management and sharing.

ƒ At the programme level, the evolution of the programme strategy (e.g. in regards to
new service areas), administrative and reporting requirements and special initiatives
(e.g. multi-country projects on specific topics (such as energy efficiency or MEAs))
all provide a different balance of drivers over time, to which individual
NCPCs/NCPPs respond as they see fit within their national set up.

120
The external factors are also diverse and define the national framework within which the
NCPC/NCPP is to operate. It appears worthwhile to differentiate at least three categories
of background factors, respectively:

ƒ State of Economy: the size of the economy, its key sectors (in particular of
manufacturing and related sectors), investment climate and national socio-economic
development priorities);

ƒ State of Environment: the natural resource endowments of the country (productive

land, seashores, forests, minerals, energy etc.) and the status of the environment,
including national environmental priorities and development status of the
environmental regulatory framework and its enforcement; and

ƒ Status of Know-how: a relatively broad category, capturing specifically the past

experience with CP (including individuals and organisations already active in CP,
type and standing of companies with CP experience and possibly government
initiatives on or related to CP, including the availability of incentives and or funding
for CP), as well as more generally the development and functioning of the ‘national
system of innovation’ (27) [55-57].

Improving the understanding of the external and internal factors at play at the national
level provides a basis for tailoring the specific national implementation strategy and
formulation of specific national outcomes and impacts and thereby increases the
likelihood of successful uptake of CP in the host country and sustainability of the
NCPC/NCPP.

5.3 Institutional Features

This section focuses on institutional and strategic features in establishment and operation
of the national centres. It covers consecutively governance (paragraph 5.3.1), focus
(paragraph 5.3.2) and operational strategy (paragraph 5.3.3).

5.3.1 Governance

The independent country evaluations found that governance arrangements could be

improved in many of the visited countries (as discussed in paragraph 4.3.2). The
importance of governance appeared to be underestimated and/or misunderstood, and as
result decision-making rules and membership categories of the highest decision-making
and oversight bodies were often sub optimal. Moreover, several NCPCs that did no longer
receive institutional funding from the UNIDO-UNEP CP Programme had abandoned their
governance structures. The self-evaluations (covered in Chapter 3) also showed that in
several countries there are more members (directors) in the highest governing board then
staff members in the Centre.

27
The term national systems of innovation is used to reflect a complex mixture of institutions (e.g. financial, legal,
scientific and technological and educational), public policies (regarding e.g. taxation; import/export promotion; science,
technology and innovation) and business and social relationships, that deliver research and technology development on new
technologies and on improving existing technologies, and bring these into widespread use.

121
In terms of the highest oversight/decision making body, different models did emerge.
Figure 5.2 contains four types on the basis of two criteria, respectively: executive only or
combined executive/non-executive and private or public-private set up.

Figure 5.2: Governance options

Board of
Board of
Executives and non- Governors (or
executives Directors
accountability
Trustees)

Management Project Steering

Executives only
Team Committee

Private
(or independent in Public – Private -
government or as NGO) Partnership

transparency

The four main governance options are:

ƒ Management Team: there is no effective external governance and all decisions,

including on strategy and budget are made by the same staff that execute the
decisions. Several NCPCs operate on this basis, either on purpose (when they are
fully independent units, e.g. as a private business (e.g. Slovakia) or an independent
business unit or centre within a larger semi-governmental organisation (e.g. China,
India)) or by default (where the NCPC did not succeed to establish effective external
governance arrangements (e.g. Sri Lanka)).

ƒ Board of Directors: a typical set up for larger private sector organisations where a
board of directors, both executive and non-executive, provides oversight to the
management team, in regard to strategy, budgets, etc. This evaluation did not uncover
any straight examples of this governance model among the NCPCs/NCPPs.

ƒ Project Steering Committee: traditionally a short-term arrangement, that primarily

oversees whether project objectives are being achieved as per planning with the
available resources. This model is also known as a Funding Board, within the
UNIDO-UNEP CP Programme typically a tripartite arrangement with membership
from host and donor governments and UNIDO, as for example currently in Vietnam,
Laos and Cambodia, and previously in South Africa. There is no long-term
membership or commitment to the operation and success of the NCPC/NCPP (i.e.
beyond the current funding period), which may explain why similar boards have
folded for those national centres that are no longer institutionally funded through the
UNIDO-UNEP CP Programme (e.g. India, China, Mexico).

ƒ Board of Governors (or Trustees) (28): a multi-stakeholder model that engages

representatives from public, private and civil sectors in defining strategy, business

28
May also be referred to as a Board of Directors, but then with Directors representing a diverse set of national
stakeholders (public and private sector), as opposed to narrowly composed Board of Directors representing only
shareholders.

122
 plans, budgets etc. for the NCPC and oversight over their implementation. When the
NCPC is not an independent organisation, but instead an isolated, stand alone (or
‘ring-fenced’) entity within a host institution, it may strictly not be possible to have
such board structure. However, using less formal arrangements and delegated
authorities, it will generally be possible to achieve a similar outcome, as with the
executive committee of the NCPC in South Africa.

Including non-executives in the highest decision making body improves accountability. A

stronger discipline is established to define a realistic business plan and achieve its
implementation on time and on budget. On the other hand, stakeholder involvement
improves transparency. External stakeholders have a say in approving strategy, and this
will generally mean that the strategy tailors to some extent to their priorities, which
makes the NCPC more relevant to them. Both transparency and accountability foster local
engagement and ownership so that it may be desirable to move to a set up with a Board of
Governors (the top right hand category in Figure 5.2). Even though this may not always
be necessary or possible, it is worth investigating ways to enhance transparency and
accountability, and share decision-making powers on direction and future of the national
centre in a meaningful way with the public and private sectors of the host country.

Regardless of the nature of the highest decision making body, common good governance
practices should be adhered to, in particular:

ƒ Increase frequency of meetings: meaningful input to define strategy, business plans

and budgets and oversee their implementation is only possible with regular meetings,
for example every 2nd or 3rd month. A lower frequency (in some countries annually or
even less) turns the board into a pseudo audit committee, that can only check whether
agreed outputs have been delivered on time and on budget, but with no opportunity
for mid term adjustment, strengthening and improvement.

ƒ Clarify decision-making rules: what board members can decide on and who has a
vote on the board. Preferably executive and administrative functions (i.e. NCPC
director and possibly UNIDO) do not interfere in board decision making by assuming
ex-officio membership. In one of the visited countries (Mozambique) there was for
example a discrepancy between memberships of board as reported by the NCPC and
as reported by the backstopping officer in Head Quarters. Elsewhere it was observed
that board members had conflicting roles that had not been sorted out (e.g. in Sri
Lanka where the chair of the board was also president of the industry association,
chair of the board of the host institution and UNIDO representative).

ƒ Size: effectiveness and efficiency suffer when boards are expanded, but some
diversity is needed to enrich decision-making. Top heavy boards were found for
many NCPCs, with up to 2-3 times more board members than NCPC staff, and most
often these would all represent the government and/or semi-governmental sector. A
small uneven number of board members generally works well for small organisations,
in case of a NCPC for example 5 or 7, all coming from different organisations and
stratified (e.g. 1/3 national government, 1/3 national private sector and 1/3 other NGO
(including donors), with an independent chair).

Many of the NCPCs that did no longer receive institutional funding from the UNIDO-
UNEP CP Programme appeared to have downscaled or even abandoned their governance
structures. Which governance structure is most likely to be effective depends obviously

123
on a number of factors and cannot be determined a-priori at programme level. An in-
depth analysis is required for each country, including an assessment of counterpart
contributions, absorptive capacity and projection of the institutional development of the
NCPC.

In addition to the decision making body, it is useful for NCPCs/NCPPs to establish an

advisory body with broader and larger membership. The aim of an advisory board is to
garner input from a variety of stakeholders and experts for strategy formulation and
review of centre performance. If approached strategically, members of the advisory board
then become advocates or champions for CP in their respective organisations, and thereby
catalyse institutional commitment to CP and NCPC. To do so, processes need to be
established so that input from the advisory board members is taken seriously and that
records are kept why some of it is acted upon and other not. To improve credibility for the
process the advisory board is preferably set up as advising the governing board and
empower the board to exercise its control over the executive management of the NCPC.

5.3.2 Focus

The UNIDO-UNEP CP Programme started with a strict focus on CP in particular for

application in manufacturing industries (e.g. pulp and paper, textile, metal fabrication,
food and agro-industrial sectors). As discussed in Chapter 2, the focus of the Programme
expanded over time, in response to:

ƒ Donors’ interests to use NCPCs as vehicle for delivery of programmes on CSR,

setting up of green credit lines, etc;

ƒ Evolving agendas in the international community and in particular in the two United
Nations agencies administering the Programme, in particular towards Sustainable
Consumption and Production (UNEP), and to a lesser extent the launch of the Global
Compact and Millennium Development Goals (both UNEP and UNIDO) and
REAP/CSR (UNIDO); and

ƒ Feed back from the Centres, including the need and/or desire to include non-
manufacturing sectors (e.g. hotels, fisheries, etc.).

The self-assessments presented in paragraph 3.3.1 demonstrated a commonly shared

interest among NCPCs in extended topics closely related to factories/plants and
technologies.

Positively, the expanded scope can be taken as evidence for adaptive management and
development of the Programme. Negatively, it can also be interpreted as ‘mission drift’
within the Programme, as the initial task of achieving widespread awareness and
implementation of CP is just starting in the host countries. This evaluation found evidence
on both sides of this argument, with perhaps a tendency of NCPCs to embrace expansion
and diversification of services in the expectation that this will enhance their financial
independence, and a tendency of national governments to prefer the NCPC to sustain a
clear focus on ‘core’ CP.

124
The recent expansions have raised some concerns:

ƒ A plethora of new terms have been added in most cases to introduce concepts or tools
that were relatively new to the Programme, but already existed elsewhere. Some
terms were invented for the Programme (e.g. CP Plus, sustainable industrial resource
management), others were incorporated from other programmes and initiatives (e.g.
SCP, CSR, Design for Sustainability etc) whilst yet others are specific examples of
funding mechanisms with much wider application (e.g. chemical leasing as one of
many applications for Performance Based Contracting (PBC), and Clean
Development Mechanism (CDM) as one mechanism under a MEA). A further
complication is that no integrative framework has been provided that links the terms,
using definitions and terminology that is accepted in the international public and
private sectors. Instead the ‘new’ concepts are now being promoted as part of an
‘integrated’ and ‘holistic’ strategy, without explaining nor clarifying what holistic
and integrated mean in relation to core CP concepts and services and how they
contribute to programme objectives and outcomes.

ƒ Due to resource constraints within national centres, new services have evolved
regularly as substitutes for, instead of additions to, existing services. Those staff that
were in the past trained in CP and gained experience through on the job learning in
CP assessments and service delivery are now withdrawn from such CP service
delivery, to be retrained in new service areas and start a new learning journey. The
benefits from their past CP training/capacity building are being compromised, as they
are not using their CP skills but instead acquiring alternative skills. The prospect of
greater CP service availability and hence greater CP uptake, which justified their past
CP training/capacity building is thus not being realised.

This evaluation found that some of the extensions could be regarded as ‘specialisation’
(improving the rigour and depth of service delivery related to CP implementation) whilst
others are better understood as ‘diversification’ (introducing services pertaining to topics
related to CP, for example SCP, CSR). This is illustrated in Figure 5.3.

Figure 5.3: Specialisation and diversification in CP

Policy
Environment,
industry, technology

Eco-Industrial
Global Compact/ Development Multilateral Policy
Millennium (and/or sustainable Environmental specialisation
Development Goals industrial resource Agreements
management)

Cleaner Production Product Applications Sustainable

Corporate Social (Life Cycle Assessment , Design for
Occupational Health & Safety Information, training and Consumption &
Responsibility assessments
Sustainability, sustainable
procurement) Production

Cleaner Production Cleaner Production Finance

Management (incl. Clean Development Mechanism,
(Environmental Management Performance Based Contracting/
Systems, Quality Management) Chemical Leasing)

Specific Process Applications Technology

(energy efficiency , hazardous waste
management, etc)
Specialisation

Technology
(development,
transfer & replication
of ESTs)

Social driven Environment driven

diversification diversification

125
The starting position for all NCPCs/NCPPs has been CP, in particular capacity
development in CP through combined training, CP demonstrations and information
dissemination and creating awareness. This is the starting point in the centre of Figure
5.3. NCPCs have specialised in two directions, along the vertical axis (North and/or
Southward) and along horizontal axis (East and/or Westward). These specialisations and
diversifications are:

1. Technology Specialisation (‘southward’): providing more detailed services on CP

implementation, financing and technology assessment and transfer. Typical initiatives
are training and advisory services on Environmental Management Systems, Chemical
Leasing, CP finance, targeted CP applications for energy efficiency, chemicals
management and/or hazardous waste management and technology assessment and
selection for transfer (including investments);

2. Policy Specialisation (‘northward’): servicing government agencies with the

development and implementation of policies and strategies conducive to CP.
Typically NCPCs have started to work in a policy advisory capacity with the agency
in the government responsible for the NCPC (in most cases the environmental or
industry department), with the possibility to branch out to other policy domains (as
the case might be for example regional development, fisheries, etc). The
NCPC/NCPP can then also get more involved in national implementation of
Multilateral Environmental Agreements;

3. Environment-driven Diversification (‘eastward’): expanding the scope of services

towards Sustainable Consumption and Production. This commonly started with
training and/or pilots on Life Cycle Assessment and Design for Sustainability,
municipal waste management, general environmental awareness initiatives for
schools and communities, and sustainable procurement for government agencies; and

4. Social-driven Diversification (‘westward’): branching out towards Corporate Social

Responsibility, in particular through factory-improvement initiatives that address
Occupational Health and Safety, community environmental health and labour
relations.

These four directions are not mutually exclusive. NCPCs/NCPPs can develop
simultaneously in different directions. However with limited resources it is generally
impossible to become a specialist provider in all areas. Therefore the NCPCs/NCPPs have
to prioritise and position themselves. This has to a certain extent happened in the visited
NCPCs, often however by default rather than by choice. This explains the diversity in
NCPCs/NCPPs that was found in this evaluation, which can then be graphically displayed
as in Figure 5.4. A more conscious and strategic approach to positioning of the Centre in
regard to diversification and specialisation options could contribute to their success and
avoid situation that limited resources are spread too thin to make a considerable impact.

5.3.3 Service Strategy

The Programme was designed to set up service delivery centres, with the clear intent for
each centre to become significant, if not leading, at the national level in the host country.
This has turned out to be unfeasible, as NCPCs had to position themselves amidst other
service providers in a growing number of countries. Some of such initiatives are
complementary and others competing, some are donor-driven (including both bilateral as

126
well as multilateral (including UNIDO projects) and others local-driven through
initiatives of government and/or private sector. The service provider model therefore had
to change gradually to accommodate national circumstances, in particular: the size of the
country and its economy; size, structure and capacity of its industrial/manufacturing
sectors; existence of a system of providers of business services (e.g. engineering and
management consultants); and/or emergence of other institutions able to deliver CP or
CP-related services.

The Programme has introduced terminology as first and second tier (or respectively sub-
national and national) centres, but there is no clear definition of such tiers and the
differences between them. The tiered system is most visible in China where at least some
35 CP Centres exist, representing each of the three layers of government, respectively
local level (city CPCs mostly involved in supervising CP audits and administering the
mandatory CP audit provisions of the China CP Promotion Law), province level
(Province Level CPCs involved in policy planning and evaluation) and national level
(China NCPC and CPCs in sector ministries and/or research institutes, involved in policy
formulation and evaluation, training, and development of technical standards). However,
even in China the role division is not commonly agreed. An alternative approach is to
consider tiers at the level of services, rather than centres. Each centre would have a
unique balance between services from the different tiers of services, instead of being
exclusively dedicated to one tier of services. As a suggestion, a three-tiered system would
be possible:

ƒ Tier 1: Audit and Training Services: advising companies and other organisations on
CP opportunities specific to their operations, and training their staff in developing,
evaluating and implementing these opportunities;

ƒ Tier 2: Development Services (policy and/or technology): undertaking enabling

activities to strengthen the policy environment for CP and increase availability of
finance and technology for CP implementation (through technology development,
assessment, adaptation and replication); and

Tier 3: Networking Services: improving communication and information exchange

between CP service providers, and providing a platform for learning, best practice sharing
and professional development (including training in e.g. CP assessments) for and among
CP practitioners.

Figure 5.4: Illustrations of current specialisation and diversification foci of selected NCPCs
Reference Cases
Starting Position for NCPC Current Programme Scope
N N

W S E W Sa E

S S

Current Status in Selected Countries

127
 China Croatia Morocco
N N N

W Sa E Sa
W E W E

S S S

Mozambique Sri Lanka Vietnam

N N N

W E W E W E

S S S

Refer to Figure 5.3. The size of the shape displays the competence areas of the NCPC. The four outlying
points are:
N = policy, resulting from policy specialisation
E = SCP, resulting from environment-driven diversification;
S= technology, resulting from technology specialisation; and
W= CSR, resulting from social-driven diversification).

Tier 1 services are delivered to organisations that can implement CP opportunities. Tier 2
services are provided to intermediaries, including government agencies, business and
professional associations, universities and providers of EST and CP services. Tier 3
services are provided for CP professionals. Most NCPCs are still predominantly
delivering Tier 1 services, while a growing number are engaged in Tier 2 services. It
appears that a niche remains for development and delivery of Tier 3 services, even though
some NCPCs already have a clear mandate for such services (in particular South Africa)
or face a demand for such services (for example China, Columbia, India).

The tiered service model can be developed nationally and/or regionally. To a certain
degree the regional roundtables for sustainable consumption and production provide a
platform for Tier 3 service delivery, but this is essentially outside of the UNIDO-UNEP
CP Programme and they have been insufficiently resourced to capture and advance best
practices. Also for Tier 2 services there are clear possibilities for international
collaboration in particular among NCPCs in smaller, neighbouring countries (e.g. East
Africa, Central America, etc), for example in development of CP standards and
technology transfer. This would enable NCPCs to specialise in selected sectors, deliver
Tier 2 services for these sectors locally and regionally, and in exchange benefit from Tier
2 services for other sectors developed by sister NCPCs in the region.

128
5.4 Service Delivery

This section discusses differences in approach among the NCPCs/NCPPs towards service
delivery in each of the five core service categories of the UNIDO-UNEP CP Programme,
respectively: information dissemination (paragraph 5.4.1), training (paragraph 5.4.2), in-
plant demonstrations (paragraph 5.4.3), policy advice (paragraph 5.4.4) and technology
transfer (paragraph 5.4.5).

5.4.1 Information Dissemination

This analysis of self-evaluation results showed that at least 80% of the responding
countries claimed to be active in regards to production of information dissemination
materials and/or delivery of awareness-type seminars (see Table 3.5). The independent
evaluations furthermore confirmed that over 80% of the visited counties had a good
portfolio of information dissemination and awareness building activities (see paragraph
4.4.1).

Throughout the Programme there is a great variety in information materials, covering

primers/mini guides, manuals, case studies, websites, fact sheets, cartoon books, videos,
etc. Likewise the formats and methods for awareness activities are quite diverse. Despite
this great variety, both within and between the NCPCs/NCPPs and the UN agencies
involved, there are no substantively different approaches in this service category.
However, a number of overall observations can be made:

ƒ A planned strategy for information dissemination and awareness creation is in most

countries insufficiently developed or missing at all. It appears that information
products and awareness events are taken on opportunistically. The justification for
each specific initiative is insufficiently developed, in terms of: specific target groups,
objectives, outcomes and desirable follow-up actions from recipients and participants;
necessary key messages, detailed content and presentation; distribution channels; and
evaluation. Similarly the relationships between information materials and awareness
activities are not sufficiently developed, for example how a mini guide relates to CP
success stories, can be used for awareness raising and is linked to technical
information sheets. NCPCs, supported by the programme management, could put
more effort in planning their information and awareness activities for maximum
impact, for example by establishing a limited number of types/categories, adopting a
common template for each type, and ensuring an ongoing stream of activities over
time. The planning for each awareness and information initiative should then feed
into a consistent communication strategy with outcome based indicators, monitoring
and evaluation tools.

ƒ The presentation styles varied considerably both within and between NCPCs/NCPPs.
There is a need to adopt a common branding and consistent use of terms and concepts
at least within each NCPC, and preferably also to some degree within the UNIDO-
UNEP CP Programme as a whole.

ƒ A considerable share of the information materials appeared to lack evidence from CP

implementation at the national level. There is an opportunity to improve the

129
 effectiveness of information dissemination and development of awareness by actively
developing more CP success stories (including post implementation evaluation of the
environmental, economic and other benefits achieved) and using these CP success
stories profoundly in all information and awareness initiatives (as visited NCPCs in
e.g. Guatemala and India are doing with videos etc.).

5.4.2 Training

For the self-evaluation, 80% of the responding NCPCs reported to be involved in

developing and delivering CP training (see Table 3.5). The review of the independent
evaluations for the 18 visited countries showed that for half of the visited NCPCs training
is a core activity in its own right with a considerable and sustained level of training
outputs over time, whilst for the other NCPCs training appeared to be more narrowly
focused and delivered only in support of other core activities (see paragraph 4.4.2).

Training programmes for CP auditors, trainers and/or other intermediaries (train-the-

trainers) are most common. Such training appears to be quite well structured using CP
methodology as the framework. The UNIDO CP toolkit [40] is commonly regarded as a
valuable resource for planning and delivering this type of training. There are some
differences among the NCPCs in regard to delivery of this auditors’ training, for example
with regard to the inclusion of a supervised CP assessment as completion criterion, the
use of case studies to illustrate application of the CP methodology and the establishment
of a register of qualified CP auditors. There is a potential to improve the training result by
targeted and selective recruitment of trainees, to ensure their qualifications and
professional roles are likely to enable them to undertake CP audits on completion of the
training. Likewise, the formulation of completion criteria is worthwhile to ensure that
registers of qualified auditors can be established, as has been formalised in China (for CP
auditors) and India (for energy auditors). However some flexibility is required for such
registers, as auditors with substantive, demonstrable experience should be eligible for
registration without having to sit through an introductory CP auditors’ training (which for
example surfaced as an issue for getting CP consultants in South Africa to register with
the NCPC).

In addition to this auditors’ training, many NCPCs deliver other training, either as
professional development in ‘advanced’ CP topics (e.g. EMS, Design for Sustainability,
Life Cycle Assessment, etc.) or as part of curricula at universities and/or schools. Some
NCPCs have developed and delivered such advanced training largely on their own, while
several other NCPCs did receive extensive training of their own staff and expert inputs in
developing new training content and programmes. This indicates a need for more
equitable access to international expert inputs for development of a balanced and
reasonably consistent system of training across the host countries. There is a tendency for
the advanced training category to be equally opportunistic as information dissemination.
In parallel with an information and awareness strategy (as discussed in paragraph 5.4.1),
it is desirable to develop a training strategy.

5.4.3 Assessment and Demonstration

Just over 80% of the respondents to the self-evaluation survey reported to undertake in-
plant CP assessments (as per Table 3.5). The comparative analysis of results from in-plant
demonstrations in the 18 visited countries showed that substantive CP assessment activity

130
is taking place in nearly 80% of these NCPCs (as per paragraph 4.4.3). It was however
also noted that follow up to in-plant demonstrations was insufficient to ascertain impacts
from in-plant demonstrations.

There are considerable differences among the NCPCs/NCPPs in how CP assessments are
undertaken. These pertain to:

ƒ Service Model: the basic distinction is between a consultancy service, in which the
NCPC or its consultants, take charge of completion of the CP assessment with inputs
from company staff, or a training and coaching service, in which the NCPC trains a
team comprised of company staff and supervises completion of the CP assessment by
the team. The training and coaching model has been adopted from the start by NCPCs
in for example Vietnam, India and China, while other NCPCs are moving towards
this approach, e.g. in El Salvador, Morocco and Costa Rica. The consultancy model
prevails in for example South Africa, Sri Lanka, Egypt and Kenya. There is a
widespread expectation that the training and coaching model is superior for achieving
actual implementation of CP (e.g. [25]), but this evaluation does not provide evidence
to support this argument. This suggests that there are also other factors at play in
determining the success of a CP assessments, as was found elsewhere in regard to
technological capability and environmental motivation of the company (e.g. [23, 58])
and choice of assessment methodology (e.g. [10, 59]).

ƒ Staffing: some NCPCs use staff members to undertake the CP assessments (e.g.
Vietnam, China), while others use only external consultants (explicit strategy in for
example South Africa) or a combination of staff and external consultants (common
model in e.g. Sri Lanka, Morocco). The external consultants are recruited from the
pool of former trainees in the respective centre’s CP audit training. This evaluation
does not provide evidence for preference either way. The use of former trainees as CP
consultants is in principle to be applauded, if managed properly. The evaluation
showed that heavy reliance on external consultants for ‘core’ CP assessment services,
can compromise the ability of the NCPC to do effective quality control for CP
assessments as it starts to lack experience and skills in CP assessments. Moreover, the
externally contracted consultants will typically have a broader environmental
consultancy background and not be equally determined to demonstrate CP as would
be expected from NCPC staff. The latter is increasingly managed by prescribing in
great detail the assessment methodology, which can however deter well established
CP consultants from undertaking CP consultancy services for the NCPC (due to
inability to use the assessment approaches they are most comfortable with).

ƒ Output: the findings from CP assessments are presented in different ways. Some
NCPCs present the findings as per the steps of the CP assessment methodology, while
others present findings with an actionable implementation plan for the business.
There are also substantive differences in the effort made to evaluate and where
possible quantify costs and benefits (economic and environmental). This applies in
particular to technology intensive options, which are just listed by some NCPCs,
whilst others have developed capacity for technology assessment and selection (see
also paragraph 5.4.5). This evaluation could not investigate the impact of the
reporting style on uptake of CP, even though based on the evaluators’ professional
judgement there is a preference for presenting the CP assessment results as an
actionable schedule of CP options with estimated costs and benefits.

131
ƒ Follow Up: there is a degree of variation in follow up to audited companies. It is
common, but not yet standard practice that the report of the in-plant assessment is at
least presented to the company in a meeting with management. Several of the NCPCs
now provide more follow up, by phone once or twice in the first couple of months
after completion of the CP assessment, or through additional site visits, depending on
logistics. Under several of the special projects (including e.g. GERIAP), follow up
was intended to result in compilation of a success story with post-implementation
results for general circulation. It is suggested to make this standard practice within the
UNIDO-UNEP CP Programme, as compiling CP success stories from CP
demonstrations would provide a good feed back on CP assessment service delivery,
provide more factual evidence for CP promotion, and be a fair request to companies
in light of the highly subsidised nature of the in-plant demonstration assessments.

ƒ Methodology: NCPCs/NCPPs start off with one type of CP assessment service,

generally referred to as a comprehensive or full CP assessment. Different
methodologies are being used for this, either a CP assessment method developed and
trialled as part of earlier CP projects (e.g. in India [60] and China [54]) or one of the
other international examples (most commonly e.g. [3, 6, 61, 62]). Increasingly the
UNIDO CP Toolkit [40] is being promoted as the preferred, or in some countries
even prescribed, methodology for CP assessments. After some local CP assessment
capacity has been created, most NCPCs start to develop simplified assessment
services, typically under the name of Quick Scans or Preliminary Assessments. This
evaluation revealed that while there is a degree of common understanding what
constitutes a comprehensive (or full) CP assessment this is not the case for the
abridged versions. Some still use a consistent CP methodology (including root source
and cause analysis and option generation) but apply this with less detailed and often
only order of magnitude data on materials, energy, waste and costs (e.g. Vietnam,
Mozambique). In other countries the Quick Scans are just lists of observations from a
quick plant walk-through (e.g. in Sri Lanka). A positive example was found in
Nicaragua where the NCPC has defined practical menus to match its service, and
necessary methodology, with company needs. Throughout the UNIDO-UNEP CP
Programme, the methodologies are predominantly engineering based (also known as
traditional) (29) ([10, 59]) and therefore rely heavily on the preparation of materials
and energy balances as the basis for generation and evaluation of CP opportunities.

It is noted that there are no attempts to develop and use localised CP methodologies that
tailor to the opportunities, capabilities and drivers of the business community in the host
countries (whilst there is a body of literature that suggests that tailoring CP concepts and
methods is key for its acceptance in different industry segments (e.g. [59, 63-67]). The
Programme relies heavily on the IRCs for CP assessment methodology and capacity
development. There is no methodological diversity among the active IRCs in the
Programme. NCPCs are therefore not exposed to alternative ways of doing CP
assessments. The IRCs’ approach therefore remains uncontested and becomes the
prescribed methodology (which is now embedded in the UNIDO CP Toolkit). Whilst this
‘engineering-based’ methodology is proven in many applications, its weaknesses are also
evident in particular when technological capability and environmental commitment in
audited companies is low, which is often the case in target companies for the
NCPCs/NCPPs. It is therefore suggested that the Programme places priority on improving
29
Alternatives to this traditional engineering based CP assessment methodologies are management systems’-based methods
(which incorporate CP in existing or to be developed management systems for environmental and/or quality management)
and quality-based methods (which originated from lean manufacturing and KAIZEN engineering).

132
CP assessment methodologies, with the ultimate aim that NCPCs/NCPPs will make an
informed choice of which method to use for a particular company (depending on its size,
sector, capabilities and commitment). Establishment of an NCPC-led Community of
Practice on CP assessment methods could be instrumental for achieving this aim.

5.4.4 Policy Advice

The survey results for the self-assessment showed that just over half of the responding
NCPCs/NCPPs were active in regards to policy advice (see Table 3.5). The detailed
comparative analysis of the 18 countries visited for the independent evaluations showed
that about 60% of these achieved a significant result in their policy advisory role.
However it was also noted that there appeared to be scope for better strategising in the
policy activities of the NCPCs (see paragraph 4.4.4).

The policy advice turned out to be partially pro-active and partially reactive (or
responsive), with the balance between both varying among the NCPCs. Proactively,
NCPCs/NCPPs have gone out and engaged with government to lobby for policy change
conducive to CP, and suggested practical ways to do so (drafted strategies, plans etc, and
made policy submissions to government). Reactively, NCPCs/NCPPs have responded to
government initiatives and endeavoured to ensure that CP was given proper consideration
in consultative processes, working parties etc., related to changes in environmental and
energy policy and legislation and national implementation of MEAs. The Programme’s
support to NCPCs/NCPPs has been very strongly focused on environmental policy
instruments (as for example reflected in the Training Kit on CP Policy [41] and to a lesser
extent in the older UNEP publications on CP Policy [9]) and training in implementation
provisions of the MEAs (for example the Clean Development Mechanism) [45]. The
Programme did not yet place priority on CP-related economic and technology policies.

The CP Policy activities by the NCPCs/NCPPs have thus been strongly focused on
environmental policy and a lesser extent energy policy, with the only profound exemption
being the work on technology transfer legislation in Vietnam. Even though this
environmental focus is understandable in light of the technical inputs provided through
the Programme, it is not properly justified in light of current insights on uptake of ESTs
and CP by manufacturing industries in developing countries. Recent work by UNIDO
[58] and others (including WorldBank [68]), has led to a heuristic model (as in Figure
5.5) for EST/CP uptake. It reflects the understanding that a company’s incentive structure
to adopt ESTs is created by three policy regimes, environmental, economic (with
subdivisions for industrial, trade and resource pricing policies) and technology. This is
transmitted to plant managers via the three pathways of governments, markets and civil
society. In turn internal plant characteristics determine the extent to which plants can
respond to these incentives [58]. A limited set of in-plant factors, market forces and
government intervention turned out to be (30) the most influential determinants for
adopting more complicated ESTs (in particular CP), and committing to higher

30
This heuristic model has been validated on the basis of a study of uptake of ESTs in 98 plants in three sectors (pulp and
paper, textile and leather/tanning) in eight countries (Brazil, China, India, Kenya, Thailand, Tunisia, Vietnam and
Zimbabwe). The findings on the perceptions of the drivers for EST adoption supported the view that various drivers across
government, markets and civil society are all motivators of compliance with environmental standards in developing
countries. Governmental pressure, either in the form of current or future regulations, and market pressure, in particular cost
competition, appear to be much more important as drivers than civil society pressure. Plant specific factors, specifically
environmental commitment, foreign (part) ownership and technological capacity, and market factors, in particular resource
pricing and technology availability, mattered significantly in determining the type of technological response, and thus in
explaining the adoption of higher order ESTs, in particular technologically complex cleaner technologies

133
environmental standards. Public intervention should therefore go beyond the traditional
domain of environmental policy and its associated implementation strategies to the use of
economic and technology policies to achieve the dual objective of reducing resource
intensity and protection of the environment [58]. This favours government intervention,
in particular to support technology-upgrading programmes and synergistic initiatives on
environment and technology policy.

The UNIDO-sponsored studies on EST transfer (but also other work) show the limitations
of the current environment-focused CP policy advice delivered though the UNIDO-UNEP
CP Programme. It highlights the need to broaden the programme’s policy focus, which
could build upon experience available elsewhere in UNIDO (and possibly other UN
agencies). More emphasis could be placed on framework conditions for technology
development and innovation (e.g. performance based funding of public sector research,
protection of intellectual property rights, fiscal incentives for businesses investment in
research and development) and productivity initiatives (e.g. support for enterprise
development service centres).

Figure 5.5: Heuristic model for EST/CP adoption by industries (source: [58])
Policies
Environmental
Economic (industrial, trade and
resource pricing )
Technology

Markets
Civil Society
Government Factor availability
Media
International donors (technology , finance,
Local communities
Regulatory agencies skilled labor and
NGOs
Technology support natural resources )
Trade and business
organisations Product (domestic,
associations
export )

Plants
Characteristics:
* Environmental commitment
* Ownership
* Profit
* Size
Technological capability

5.4.5 Technology Transfer

The survey responses for the self evaluation showed that just under 50% of the
NCPCs/NCPPs claimed to be active in regard to EST transfer (see Table 3.5). The
comparative analysis of technology transfer results in the 18 countries visited for the
independent evaluations revealed highly different expectations on what is being covered
by technology transfer. It also revealed that nearly 80% of these visited countries had

134
activities that could be regarded as supportive of technology transfer, even though in
many cases the NCPC itself would not qualify these as such (but rather a spin off from
CP technical standard setting or extensions of CP assessment activity) (see paragraph
4.4.5).

It is noted that even though some praiseworthy results are being achieved by some
NCPCs, overall the technology transfer initiatives within the UNIDO-UNEP CP
Programme lack strategy and focus. This is largely attributable to the absence of clarity
on terms and scope, as the Programme has not attempted to define technology transfer or
elements of successful EST transfer. It is possible to improve this situation by building
upon the excellent work done elsewhere in regard to the provisions for technology
transfer under the MEAs. In particular:

ƒ The Intergovernmental Panel on Climate Change (IPCC) prepared an extensive

review of methodology and policy for technology transfer [69]. It defined:
“Technology Transfer is defined as the broad set of processes covering the flows of
knowledge, experience and equipment amongst different stakeholders, such as
governments, private sector entities, financial institutions, Non Governmental
Organisations (NGOs) and research/educational institutions. The broad and
inclusive term ‘transfer’ encompasses diffusion of technologies and technology
cooperation across and within countries. It comprises the process of learning to
understand, utilise and replicate the technology, including the capacity to choose it
and adapt it to local conditions (pg 55, [70])”.
ƒ
There is broad consensus (e.g. [69, 71, 72]) that the transfer of technology
follows a number of distinct stages, regardless of the specific pathway. An
integrated model comprising five stages is presented in Figure 5.6. These stages
are: assessment (identification and selection of technology, potentially including
elements of technology sourcing and technology creation); agreement (terms,
conditions and modality of transfer); implementation (execution of the
technology transfer); evaluation and adjustment (learning and continuous
improvement); and replication (widespread use of the transferred technology).
The stakeholders involved and the specific decisions and actions taken at each
stage differ greatly depending upon the pathway. By analysing the interests and
influences of different stakeholders at each stage it is possible to determine how
various challenges in technology transfer can be effectively addressed.

135
Figure 5.6: Main stages of EST transfer and development of its benefits
(Source [72]: integrated from [70] and [71])

Technology Creation Technology Sourcing

Design & Modification
Testing

Development ASSESSMENT Identification

Te ch no lo gy Tra ns fe r
Testing &
Innovation

P ro c es s
Technology selection Evaluation

AGREEMENT

Modality, terms and conditions

IMPLEMENTATION

Technology delivered
learning
EVALUATION &
ADJUSTMENT

Technology optimised

REPLICATION
Cumulative benefit
(socio-economic and environmental )

Using Figure 5.6 as reference point, it is noted that the current programme efforts for
technology transfer are geared towards the first stage, assessment, particularly towards
technology sourcing, through activities like benchmarking, technology gap analysis and
technology identification. This is most profound in for example Vietnam - the NCPC
receiving extensive programme support for technology transfer. There is also some
activity in regard to evaluation, adjustment and replication, but this is initiated locally at
the national level by the NCPC and not yet acknowledged at the programme level, in
particular in India, where the NCPC is hosted by an organisation with a strong track
record in technology up-grading. Overall, it is clear that adopting current leading insights
in EST transfer could result in a more balanced and integrated set of programme activities
on EST transfer within the UNIDO-UNEP CP Programme.

5.5 Portfolio and Network Management

The previous sections discussed the diversity within the portfolios of the NCPCs/NCPPs
both in regard to their institutional arrangements (including governance, focus and service
model/strategy) and their approaches to delivery of each of the five key services. With the
widening scope of CP activities in the Programme, it is not possible for each NCPC to
claim expert status on all aspects of the programme. The human, financial and other
resources are not available for doing so (31). It is therefore strongly suggested that the
UNIDO-UNEP CP Programme supports each centre to position itself taking due
consideration of the national status of CP implementation (including activities of other
actors), national socio-economic development and environmental protection priorities and
technological capability and environmental commitment of key manufacturing sectors.
This positioning considers both the focus (in light of the discussion on diversification and
specialisation in paragraph 5.3.2) and service model (with regard to a split between Tier
31
Even though it could be argued that these limitations could be addressed by increasing the funding of the centres, but
given that in current situation many of the NCPCs already do not manage to spend the allocated finances in the agreed
timeframes (e.g. Mozambique, South Africa, Sri Lanka, etc.), it is unlikely that increasing funding can substantially
alleviate the resource constraints.

136
1, Tier 2 and Tier 3 services as discussed in paragraph 5.3.3). Effective positioning will
benefit from increased accountability and transparency of decision making for each
NCPC, as per the discussion on governance (in paragraph 5.3.1), which in turn can also
bolster local ownership of the NCPC.

The process of national positioning would result in NCPCs that display different balances
between Tier 1 (audit and training services), Tier 2 (development services) and Tier 3
(networking services). The network of CP Centres would then evolve as conceptually
displayed in Figure 5.7. CP Centres predominantly providing Tier 3 services would
service several other CPC’s that are predominantly providing Tier 1 and/or Tier 2
services. This could be on a national basis within the large countries (e.g. China, South
Africa, Brazil) or on a regional basis among smaller countries (for example in Latin
America). The CP Centres providing predominantly Tier 2 services would service a
number of CP Centres that predominantly provide Tier 1 services, and collaborate with
other CP Centres that also provide predominantly Tier 2 services, but for example in
other policy or technology areas. There is no need to limit the number of providers of Tier
1 services, as this would ultimately be determined by the size of the CP market.

Similarly, the strategic positioning of the NCPC in regard to focus of its activity area,
would lead to CP Centres that have a different blend of activities on diversification
(socially and/or environmentally towards CSR or SCP respectively) and specialisation
(towards policy and/or technology). As illustrated for six countries in Figure 5.4, this is
already happening. It can be further strengthened, and would then lead to a network of CP
centres with diverse foci, as illustrated in Figure 5.8.

Figure 5.7: Conceptual outline for the network of CP centres based on diversified service models

Tier 3
(Networking)
Tier 2
(Development )
Tier 1
(Audit and training )

137
Figure 5.8: Conceptual outline for the network of CP centres based on diversified foci
N

W S E

W E

N S

N W S E

W S E

W Sa E
N

N
W E S

S Sa
N W E

W E
S

N
N

W Sa E S

N
N
W E
W S E

N
S

W E
W E
S
S

W Sa E

S N
S

W E

W E

It is likely that the strategic positioning of each CP Centre would change over time in
response to changing national circumstances (e.g. in macro-economic conditions, national
priorities and emergence of other providers of CP or CP-related services). The positioning
can therefore be reviewed as part of e.g. 2 or 3-yearly forward business planning cycles.
However once a position has been determined, some discipline is required to adhere to it,
to avoid drifting back to opportunistic operation in which human, technical and financial
resources are spread thinly at the detriment of quality and ultimately impact of service
delivery and recognition and status of the respective NCPC.

This tailoring of the NCPC and its activities to the local content is a process that needs
strategic support through the Programme, in addition to the predominant technical and
operational support provided so far through the IRCs. Diverse NCPCs will then coexist
which will pose further challenges to programme management. A change of the funding
model is required to manage the diversity among NCPCs (eventually including other CP
Centres not established under the UNIDO-UNEP CP Programme). The funding could be
split in at least two categories, respectively:

ƒ Block Funding: guaranteed funding on a country-basis to selected NCPCs to establish

core capacity in CP, and enable planning and institutionalisation of the NCPC as a
local CP institution. This is similar to the current institutional funding model. As in
the past, the source of this block funding would be country specific project
agreements between UNIDO, the host country and at least one donor country.

138
ƒ Competitive Grant Funding: funding budgets for targeted activities, regardless of
specific location, available on a competitive basis for NCPCs and other CP Centres
meeting predetermined standards and conditions. The NCPCs (and eventually also
other CP Centres not established under the UNIDO-UNEP CP Programme) would
compete amongst each other for access to funding from this programmatic funding
pool and this would then be made available against specific deliverables. This is
similar to some of the past funding provided through UNEP’s multi-country projects
to which some of the NCPCs have contributed, including for example the GERIAP
project on energy efficiency through CP in Asia Pacific. The source of this
competitive grant funding would be programmatic funding provided by one (or
possibly several) donor to specific activities within the NCPC programme (for
example on Design for Sustainability, etc.).

In the start up phase the NCPC would be largely, if not exclusively, funded with block
funding, and at this stage it would be unlikely that the new NCPC could successfully
compete for programmatic funding. In a second phase, the block funding would reduce,
and the NCPC could complement this with competitive funding from programmatic
sources. As time progresses, the block funding could completely be phase out. The block
and competitive funding through the UNIDO-UNEP CP Programme would be
supplemented by fund raising locally by the NCPC, including grants from national
government, project related funding from other bilateral or multilateral donors and fee-
for-service (e.g. training, auditing etc.). This funding approach is illustrated in Figure 5.9.
The figure includes as the last phase an independent NCPC that does not receive any
block funding through the UNIDO-UNEP CP Programme. This would be the target
situation for the NCPCs established by the Programme, but it could also be viewed as the
model by which CP Centres not established by the Programme could participate in the
Programme (and possibly receive some competitive grant funding for programmatic
activities).

Figure 5.9: Schematic presentation of the funding model for NCPC over time

NCPC fee for service

(directly by NCPC, from
Funding national government, aid
projects, private sector
etc)
competitive grant
(programmatic funding
through UNIDO, UNEP, etc)

block grant
(institutional funding
through UNIDO)

Phase 1 Phase 2 Phase 3 Independent

Time since
establishment

It is strongly suggested that the introduction of programmatic funding be accompanied by

appointment of capability leaders in the programme management. These capability
leaders would be in charge of programmatically funded activities in multiple countries.

139
Their main responsibility would be to ensure delivery of programmatic activities, quality
control and effective dissemination of the results, experiences and lessons learned to all
CP Centres in the Programme. This would lead to a matrix management structure for the
Programme, with national project managers, having responsibility for the block funding
to selected countries, and the capability leaders. Some capability leaders could be
positioned in the UNIDO programme management unit, others might be found in other
UNIDO units (e.g. POPs, energy, water). UNEP could also provide capability leaders for
some topics related to sustainable consumption and production, and possibly other United
Nations agencies for other topics (for example International Labour Organisation for e.g.
Occupational Health and Safety). Moreover it could also be considered that senior staff
from some NCPCs would assume a capability leadership role.

The changes discussed above involve a qualitative change in the Programme, as it would
turn from the current project-by-project mode increasingly into a network-managed
approach. The activity to establish NCPCs in developing countries with a substantive
industry basis is largely completed, certainly if consideration is given to comparable CP
centres that have been established by other donors or international programmes in other
counties. The share of the block grants, for establishment of the NCPCs, in the total
budget (both at Centre and Programme level) will therefore diminish, with the greater
share becoming programmatic funding, for ongoing professional development and
strengthening of existing NCPCs. The question is then which NCPCs and/or other CP
Centres can undertake activities funded by the Programme and/or participate in the
networking activities. This can in principle be done via a set of Memoranda of
Understanding, on a one-on-one basis between a CP Centre and the UNIDO-UNEP CP
Programme management. However, a transparent and inclusive process would be
preferable in particular to engage CP Centres that have not been established by the
Programme, and achieve maximum leverage from the programme’s networking activities
among all CP Centres that commit to the Programme’s mission and aims.

One possible way to implement this would be to support the establishment of an

association of CP Centres, for which several examples exist but with different niches, e.g.
the association of Pollution Prevention Programmes in the USA (www.p2.org) and the
Regional Network of the Word Business Council for Sustainable Development
(www.wbcsd.org). The association of CP Centres would establish its statutes, and define
membership criteria. Any CP Centre that would like to join, could apply, and would have
to demonstrate that it meets the membership criteria. Membership could be time-bound,
so that after say 2 or 3 years any member would have to re-apply for membership to
demonstrate its ongoing ability to meet the eligibility criteria. The networking and
programmatic activities of the UNIDO-UNEP CP programme would then be delivered to
members of the association.

It is strongly suggested that the Association would introduce various categories of

membership. These could be:

ƒ Ordinary member: CP Centres that are aligned with the aims and objectives of the
UNIDO-UNEP CP programme, and fulfil a demonstrable public interest role in
promoting CP and related topics. Ordinary members would be entitled and sponsored
to participate in network meetings (training and other professional development,
NCPC Directors’ meetings, etc). They would also have to provide regular inputs to
the Programme (e.g. new case studies, lessons learned, etc), in exchange for right to
use UNIDO and/or UNEP logos and endorsements.

140
ƒ Associate member: CP centres (or other organisations, including e.g. government
agencies, non-governmental organisations, private sector consultants and/or
individual professionals) that have an interest in CP promotion but do not fulfil a
demonstrable public interest role in promoting CP. Associate members can participate
in the public information sharing, for example through regular newsletters, access to
publications, etc. They cannot participate in network meetings, unless specifically
invited to present an element unique or new to the Programme. They also would not
get the right to use UNIDO and/or UNEP logos and/or endorsements.

ƒ Programme Member: CP Centres that consistently meet high professional standards,

and in their mission and business plans are exceptionally aligned with the aims and
objectives of the UNIDO-UNEP CP Programme. The Programme Members have
more rights and obligations than the ordinary members, including the right to
compete for programmatic funding. It may be necessary to differentiate the
programme member category further as the programme activities could require quite
different skill sets (e.g. for specialist work on technology or on policy).

It might be considered to establish further membership categories for example of

founding members and/or sponsors (donors and possibly private sector contributions). For
illustration purposes, Table 5.1 contains preliminary suggestions on how the membership
criteria could be set up for Ordinary members only. Developing a full and balanced set of
membership criteria for a restructured network of NCPC and other CP Centres was
however well beyond the scope of this evaluation. It is therefore suggested that a follow
up project be launched for the further development of the proposals made here for revised
network management strategy and associated membership criteria.

Table 5.1: Some starting suggestions for membership criteria for association of CP Centres
Eligibility Criteria Obligations Benefits
Category: Ordinary Members
Independence: CP Practice Information Access:
¾Own business plan, budgets, ¾Report annually on CP ¾x hrs/yr from helpdesk
reports and board practice, and keep ¾newsletters
¾Effective government and records in auditable ¾member contacts
industry representation in board manner ¾access to databases
¾CP identified as a core service ¾Annual review ¾access to training
area (for example being a meeting with CEO of ¾access to thematic
signatory to International CP network regarding working groups
Declaration) expectations and (Funding rules to be
¾Not-for-profit operation outcomes from ascertained)
¾Code for fair-trade membership
¾Ensure flow on benefit
from membership to
stakeholders
Track Record: Participation: Business development
¾CP service delivery (training, ¾Regular attendance at ¾Assistance for execution
assessment, information designated networking of national assessment
sharing, advocacy etc) meetings of CP status, and
¾Public benefit (networking ¾Evidence of sharing of positioning of the CP
nationally and internationally, experience, knowledge Centre therein
funded and non-funded) etc into the network
¾Professionalism and ¾Be an ambassador for
accountability (preferably the network

141
 through ISO 9000/1400 for CP
service delivery
Application Process: Recognition:
¾Provide evidence from past 3 ¾Rules to be established
years for peer review by for use of UNIDO
membership committee at and/or UNEP logos
application stage, to be renewed
after 3 years

142
6
Programme Assessment
_______________________________________

6.1 Introduction

This chapter contains the overall assessment of the UNIDO-UNEP CP Programme by the
international evaluation team. It integrates the findings from the three constituent parts of
the evaluation study, respectively the programme review (as reported in Chapter 2), the
self-evaluation (as reported in Chapter 3) and independent country evaluations (as
reported in Chapter 4. It also takes into consideration the analysis of differences that were
found among the NCPCs/NCPPs (as covered in the portfolio analysis in Chapter 5).

This global programme-level evaluation was structured around four primary and two
secondary evaluation criteria. The primary criteria relate to the uptake of CP, and were
relevance, effectiveness, efficiency and sustainability. The secondary criteria assessed the
performance of the UNIDO-UNEP CP Programme as a development assistance
programme, in particular for capacity development and ownership. The primary
evaluation is covered in Section 6.2, and the secondary evaluation in Section 6.3. Section
6.4 summarises the main findings from the programme assessment.

6.2 Uptake of Cleaner Production

The primary evaluation assessed the success of the UNIDO-UNEP CP Programme in

achieving uptake of CP practices, methodologies, technologies and policies by
businesses, the private sector, government, academia and other relevant stakeholders in
the host countries. This success is determined by relevance, effectiveness, efficiency and
sustainability of the Programme’s activities. These four criteria are closely related: If the
Programme is relevant for the country and its stakeholders, it stands a good chance to be
effective, because it will have the support of the main stakeholders. The programme is
judged to be sustainable if it is likely that its present positive results (effectiveness) will
continue into the future. Finally, given that programme resources, both human and
financial, are limited, an efficient use of these resources will enhance the possibility of the
programme to be effective, that is to achieve positive results.

6.2.1 Relevance

Relevance is concerned with the applicability and value of the programme elements (i.e.
the CP concept, the CP services, the NCPC institution, the global network and the
technical assistance inputs) for the intended beneficiaries (i.e. the private sector,
government, academia and research institutes in the host country).

143
The result of the programme level assessment on relevance is presented in Figure 6.1. It
shows that the relevance of the UNIDO-UNEP CP programme is satisfactory.
Specifically, it can be seen that:

ƒ Among the target beneficiaries, relevance is highest for government, followed by

private sector (except for the CP concept), and then academia.

o Relevance for government is good due to the alignment of CP with the entering
into force of MEAs and ongoing trade liberalisation and economic reform. CP
also becomes more urgent with increased industrial pollution and resource use,
despite this not yet being a national priority in the host countries with a relatively
lower level of industrialisation.

o Relevance ranks second highest for the private sector, largely on the basis of its
economic merit for businesses. The technical potential for CP is high due to
performance gap between commonly used technologies in developing countries
and international best practices [58]. CP

Figure 6.1: Programme-level assessment on relevance

Programme Scores: 1 = unsatisfactory, 2 = low, 3 = satisfactory, 4 = good and 5 = excellent

Programme Assessment: Relevance

4
programme score

0
CP Concept CP Services NCPC Institution Networking Technical Assistance

Private Sector Government Academia

has been sold to the local stakeholders on the basis of a ‘win-win’ premise,
claiming that CP would merge environmental and economic benefit. This premise
can however not universally be achieved in developing countries, largely because
compliance initiatives for environmental regulations lag behind. This evaluation
found that the inability of NCPCs to demonstrate universal, clear cut, win-win
examples of CP has somewhat weakened the buy-in from enterprises and private
sector stakeholders.

144
 o Relevance ranks third highest for academia. It is being recognised that CP can
add value to teaching and research, but no programme element is specifically
targeted at opportunities for, and/or needs of, academia and research institutes. In
some countries the NCPC has however developed specific programmes to service
academia, in particular in their education programmes. This is particular the case
for NCPCs that are hosted by universities, for example in Vietnam, Nicaragua
and Mexico.

ƒ On average among all stakeholders there is hardly any difference in relevance

between the different programme elements. However, some programme elements (in
particular the NCPC institution, CP concept and the CP services) are markedly more
relevant to government than to the other stakeholders (in particular academia).
Specifically, in regards to each programme element it can be noted that:

o CP Concept: continues to be understood differently by different stakeholders in

the Programme, and at the national level among beneficiaries of the
NCPCs/NCPPs. It would appear that this situation is not helped by the expanding
focus of the Programme, in particular the diversification (towards CSR and SCP)
and to a lesser extent the technology and policy specialisations (as discussed in
paragraph 5.3.2).

o CP Services: the Programme’s focus on capacity building through delivery of

assessment, training and information services is generally supported. More
specific services are however needed to create an enabling environment for CP
and as a result thereof a sustainable demand for CP services. These could for
example include support for establishment of a national system of business
advisory services, strengthening of vocational training, Research and
Development initiatives, etc.

o NCPC Institution: while governments in the host countries seemed to be attached

to the notion of having a dedicated CP Centre as demonstrable evidence of
environment and sustainable development policy, the business sector and
academia view the NCPC more as a means for service delivery (and not an ‘end’
in itself).

o Regional and Global Networking: networking is generally considered relevant for

learning, professional development and information sharing in and between the
NCPCs. The relevance of networking is so far largely hypothetical due to low
networking intensity, leaving expectations of programme stakeholders unmet.

o Technical Assistance Inputs: are considered relevant and essential for up-skilling
the NCPCs/NCPPs in particular in its early establishment stages. The current
and/or past levels of technical assistance received by many of the NCPCs are
however low. Expectations for technical assistance are in those countries
currently not being met, in particular not for academia.

6.2.2 Effectiveness

Effectiveness addresses whether the design of the programme (i.e. national centres, global
management and networking, and technical assistance) and its implementation enable the
Centres and beneficiaries to achieve the programme’s intended results (i.e. uptake of CP).

145
Figure 6.2: Programme-level assessment on effectiveness
Programme Scores: 1 = unsatisfactory, 2 = low, 3 = satisfactory, 4 = good and 5 = excellent

Programme Assessment: Effectiveness

4
programme score

0
Programme Management National Centres Technical Assistance Networking

Figure 6.2 shows the assessment result on effectiveness. Overall effectiveness is rated as
moderately satisfactory. As evidenced by the independent national evaluations, the results
varied considerably between NCPCs and between different components of the
Programme. Specifically, for each of the main programme elements, the following can be
observed:

1. Programme Management: this covers the programme strategy, liaison with donors
and programme stakeholders (including UNIDO UNEP collaboration), planning and
reporting, budget and financial control and mentoring. Across these elements the
effectiveness was rated relatively low, because:

- Programme goals are ambitious and not consistently included in national

projects, lack an institutional dimension for NCPCs and are weakly linked to
activities. The ongoing diversification and specialisation with CP+, SCP
and/or CSR distracts from initial objectives. Moreover, the initial
expectations that the Programme would contribute to creating national
markets for CP services and decoupling of economic growth from
environmental impact have not been integrated in the Programme’s design
and strategy.

- Outcomes have not been appropriately defined and/or have been mixed up
with outputs, and can therefore not be appropriately monitored for strategic
and adaptive management of the Programme and customisation to national
needs;

- National contexts are insufficiently analysed and NCPC service areas are
therefore insufficiently customised to national needs;

146
 - The CP capabilities available for Programme Management were insufficient
to oversee the development of the Programme, in particular of its strategy and
focus, and ensure their consistency with mainstream and evolving CP
methods, policies and tools and alignment with key policy and industry
developments;

- The Programme management unit did not have the necessary human
resources to claim thematic leadership in the international CP community and
coach the NCPC Directors. It could also not ensure that best practice was
being applied in business planning, communication and service delivery by
the NCPCs;

- Contact with NCPCs no longer institutionally funded through the UNIDO-

UNEP CP Programme has become sporadic; and

- Lack of unified direction among the UN programme stakeholders and donors.

2. National Centres: these are the national institutions created through the Programme
(as NCPCs or NCPPs) that deliver CP services, in particular information
dissemination, training, CP assessments and in-plant demonstrations, policy advice
and technology transfer services. Across this category, the Programme’s performance
was rated as satisfactory, because:

+ NCPCs do reach their target groups and implementation of low/intermediate

technology options takes place in selected companies with some, albeit
significant, contributions to economic development, resource conservation and
environmental protection;

+ The effectiveness of participatory delivery of CP assessment services (training

and coaching of company assessment teams) is perceived to be higher than for
consultant-driven CP assessments;

- Service delivery is not based on rigorous planning and feasibility studies which
identify the demand for CP services by different enterprise groups and other
clients and their willingness to implementation. Service delivery therefore
becomes opportunistic, lacking strategic planning and targeting, which reduces
effectiveness. There is also no consistent evidence for the application of best
practices in the areas of communication, advocacy and stakeholder engagement,
professional and vocational training, and CP auditing;

- NCPCs are not yet effective as catalysts for innovation, as they do not yet achieve
substantive EST transfer or initiate R&D for CP. Several factors could be at play,
e.g. selection of client industries that have insufficient financial, technical and/or
managerial resources to innovate or assimilate innovative technologies from
elsewhere, insufficient technological acumen of the NCPC and/or lack of
supportive policy framework;

o Within their national contexts, NCPCs appear to be able to contribute to, albeit
not drive, the diversification of the CP agenda into SCP and CSR; and

147
 - There are varying degrees of mission drift away from CP service delivery by the
NCPCs, in particular after ending of their institutional funding phase through the
UNIDO-UNEP CP Programme (but also before).

3. Technical Assistance: this covered the provision of specialist technical,

methodological and policy assistance (from international consultants), training of
NCPC staff, production and distribution of resource materials and the establishment
of an award scheme for CP practitioners. Across these components, the effectiveness
was rated largely satisfactory, specifically because:

+ The technical assistance provided has in most country been effective in equipping
the NCPC with the technical and methodological skills and resources to
undertake CP training and in-plant assessments;

o Only some NCPCs have received substantially more technical assistance to

support them in undertaking policy and technology transfer initiatives, and

- NCPCs that are no longer institutionally funded through the UNIDO-UNEP CP

Programme do not receive any ongoing support to improve, or even just retain,
their core CP competencies.

4. Networking: this covers activities aimed at improving collaboration, learning and

information exchange between NCPCs/NCPPs in different countries, currently
mainly through the ‘annual’ meetings of the NCPC/NCPP directors, regional
cooperation initiatives and network promotion. For these components the
effectiveness was rated low, specifically because:

- The networking intensity is low and learning and exchange between NCPCs has
not yet been achieved to a significant extent, nor has access to specialist CP
technology information been provided;

+ Effective collaboration between NCPCs in the same region has been achieved on
project basis (e.g. GERIAP) and through (sub-) regional networking (e.g.
LatinNet); and

- NCPCs that no longer receive institutional funding through the UNIDO – UNEP
CP Programme are no longer aware of activities and developments in the
Programme and operate independently and may no longer contribute significantly
to the aims and objectives at the programme level.

6.2.3 Efficiency

Efficiency is concerned with the allocation of available resources in order to achieve

optimal benefit from the UNIDO-UNEP CP Programme. The key variable is efficient
service delivery to the NCPCs (in regard to programme management, technical assistance
inputs and networking) and through services of the NCPC (i.e. its training, information,
assessment, policy and technology services) to target beneficiaries in the host country
(including businesses, private sector, government and academia).

148
Figure 6.3: Programme-level assessment on efficiency
Programme Scores: 1 = unsatisfactory, 2 = low, 3 = satisfactory, 4 = good and 5 = excellent

Programme Assessment: Efficiency

4
programme score

0
Programme Management National Centres Technical Assistance Networking

The result of the programme assessment on efficiency is displayed in Figure 6.3. This
shows a mixed result on average about satisfactory. The programme scores for each
programme element are justified on the following grounds:

1. Programme management: this refers to the central, agency led management strategy
for the UNIDO projects that constitute the core of the UNIDO-UNEP CP Programme.
Its efficiency was rated low, for the following reasons:

- High administrative burden and micro-management for financial control

and reporting has not left enough time and resources for strategic
management of the Programme;

- A number of systemic constraints inherent in modalities available for

UNIDO to fund and implement technical cooperation initiatives
(including agency led execution and centralised programme management
from headquarters); and

+ Some evidence of attempts for adaptive management with proposals for

new service areas, through diversification and/or specialisation. These
have been donor-initiated but with endorsement of some NCPCs.

2. National Centres: this pertains to efficiency of creating a CP service-delivery entity

within an existing host organisation. The efficiency of national centres was rated
moderately satisfactory, on the following grounds:

+ Some evidence that NCPCs are starting to standardise service delivery and
thereby improve the quality and efficiency of their existing services;

149
 + After an initial period of adjustment, most NCPCs succeeded to achieve mutually
beneficial working relations with their host institutions, and benefit from the
availability of specialist skills and resources of their host institution (e.g.
laboratory facilities, legal and administrative functions, marketing and
communications, ICT, etc);

- Host institution commitments for in kind provisions to the NCPC have however
not materialised in several countries, and as a result resources from the
Programme have been diverted to compensate for the lack of inputs from the host
institution;

- Also in a few countries the host institution has continued or started delivery of
competing CP and/or CP-related services (and/or services very closely related to
CP, e.g. training and advisory service on Environmental Management Systems,
Waste Minimisation, Energy Efficiency, etc);

- A degree of duplication was uncovered in several of the visited NCPCs due to a

lack of consistency in concepts, methods, styles etc. within and between the areas
of service delivery. In some instances this inconsistency has been created by the
need to apply international standard approaches in multi-country projects instead
of existing national concepts and methods (e.g. GERIAP Project on industrial
energy efficiency in Asia Pacific countries); and

- Some NCPCs rely heavily on outsourcing to external consultants for CP

assessments, delivery of CP training and preparation of CP information materials.
These are faced with the challenge to maintain core CP competencies in the
NCPC or would otherwise become a project management unit with limited
capabilities for effective quality management.

3. Technical Assistance: the provision of international expert inputs has been largely
organised through, and/or on behalf of, a small group of International Reference
Centres (IRCs), currently only from Austria and Switzerland. The efficiency of this
arrangement has been ranked as moderately satisfactory, for the following reasons:

+ Those NCPCs that have received substantive and regular technical assistance
from any or several of the International Reference Centres have generally
benefited from assistance provided for their initial establishment and building
technical and methodological capacities in particular for undertaking CP
assessments;

- The lack of influence of NCPCs on the choice of CP consultants has been a

concern for NCPC directors since the Programme establishment. Moreover the
absence of diversity among the key IRCs and lack of competition with other CP
service providers compromises effective quality control over the IRCs, and hence
the programme’s efficiency; and

- Reportedly high administrative burden for contracting and providing technical

assistance inputs.

4. Networking: the networking and cooperation between the NCPCs/NCPPs in different

countries is currently being organised and facilitated by the Programme’s

150
 management unit of UNIDO with some contributions from UNEP. The networking
efficiency has been ranked low, for the following reasons:

+ Publications are consistently considered useful, but not widely known nor
generally used;

+ Positive experience from regional cooperation among NCPCs and other CP

service providers through regional projects (in particular GERIAP in Asia) and
regional networking initiatives (in particular LatinNet);

- Meetings have been irregular (including the ‘annual’ meeting of Directors), and
follow up on meeting outputs has been slow or not forthcoming at all (for
example on regional cooperation project proposals and NCPC criteria);

- The dominant country-based funding model has not catered for development of
programmatic networking activities; and

- Networking has been perceived as centrally driven, without sufficient

consultation on networking needs of NCPCs and ways to meet these;

6.2.4 Sustainability

The last of the primary evaluation criteria is sustainability. It deals with the probability or
likelihood that that the benefits achieved from the programme will continue into the
future, with a particular focus on the availability of CP services, the environmental and
productivity benefits in industry, and the catalyst role for sustainable industrial
development.

The assessment on sustainability of the programme is primarily justified by expected or at

least likely trends in seven determinants that constitute an enabling environment for CP
uptake in the host countries. These seven are:

1. Willingness of target industries, governments and/or other organisations (including

current and potentially other donors) to pay for the provision of CP services;

2. Continued availability of the know-how and skills to deliver high quality and
effective CP services;

3. Consensus about the relevance and benefits of CP (‘critical mass’);

4. Presence of framework conditions conducive to CP (e.g. legislative framework,

policy, tax, financial incentives, etc.);

5. Technology push (availability of new CP technologies and practices customised to

local industry needs and capabilities);

6. Market push for CP (through prices for water, energy, waste, materials, etc.); and

7. Market pull for CP (exerted through the supply chains that the target industries are
part of or would aspire to become part of).

151
Table 6.1 provides an indication of the expected trend in each of these key determinants.
This shows that three key determinants will continue, namely framework conditions,
market push and market pull. However, only in a few countries substantive change has so
far been achieved, so that these determinants will continue at a low level in most
countries. Two determinants have not yet been achieved respectively the willingness to
pay and technology push for CP. The other two determinants are likely to gradually
decline over time if the UNIDO-UNEP CP Programme were to be discontinued,
respectively know-how/skills and critical mass.

The overall outlook for sustainability of the programme achievements remains however
relatively good, as the benefits that have already been realised are unlikely to be
discontinued, even though expansion of these benefits to other potential beneficiaries may
not materialise. This is displayed in Figure 6.4, which shows that across the benefit
categories the sustainability is between ‘good’ and ‘low’.

Table 6.1: Trends in enabling environment for CP uptake in absence of the UNIDO-UNEP CP
Programme
Determinants Likely Development in Justification
Absence of UNIDO-
UNEP CP Programme
1. Willingness Not Yet Achieved ¾ Willingness to pay for CP services has not yet been
to Pay achieved except for larger, internationally-oriented
companies in some countries, and this situation is
likely to continue
2. Know- Declining ¾ Know-how and skills of experts trained in CP will
how/Skills initially remain, but in absence of continuing capacity
building and learning opportunities, their quality and
effectiveness are likely to decline over time
3. Critical Mass Declining ¾ While some critical mass for CP may have been
achieved in several countries, it is expected that this
will decline over time in absence of CP advocacy
4. Framework Continuing ¾ The CP-fostering changes in government policy and
Conditions other incentives will continue. This has however only
been achieved in a few countries
5. Technology Not Yet Achieved ¾ Availability of CP technologies and products has not
Push been increased by the Programme
6. Market Push Continuing ¾ Improvements in resource pricing will continue. This
has however only been achieved in a few countries
7. Market Pull Continuing ¾ Market pull for CP will remain limited to larger
companies with international orientation (e.g. in
ownership or markets)

152
Figure 6.4: Programme-level assessment on sustainability
Programme Scores: 1 = unsatisfactory, 2 = low, 3 = satisfactory, 4 = good and 5 = excellent

Programme Assessment: Sustainability

4
programme score

0
Availability of CP Services Productivity Gains and Environmental Catalyst Role for Sustainable Industrial
Improvements Development

In particular, the programme assessment found:

1. Availability of CP Services: the sustainability is rated as satisfactory, because:

o Former consultants and trainees from the Programme will continue to deploy
their CP know-how and skills in their service delivery, but likely to be integrated
in other services (e.g. general environmental or management consultancy); and

o NCPC as service delivery organisation could disappear, or by financial

considerations be driven into non-CP service delivery areas.

2. Productivity Gains and Environmental Benefits: the sustainability of these benefits is

rated as good, because:

+ Businesses and other organisations that have implemented CP, will continue with
implementation as they are achieving real time environmental and/or productivity
benefits; and

- In the absence of a continued drive for CP, the productivity gains and
environmental improvements are unlikely to expand further.

3. Catalyst Role for Sustainable Industrial Development: the sustainability for this
benefit category is rated low, because:

+ Catalytic effect could continue at least in the short term as information and other
materials produced by NCPC remain available for CP advocacy; and

153
 - Public interest functions of the NCPC are however likely to disappear first
(information dissemination, training and policy advice).

6.3 Capacity Development and Ownership

The secondary criteria assess the success of the CP Programme as a development

assistance intervention, in particular capacity development and ownership. There is also
some parallel between the secondary and primary evaluation criteria. Capacity
development is mostly related to effectiveness, whereas ownership is principally
influenced by relevance and sustainability. Capacity development and ownership both
relate fundamentally to quality of project implementation and are assessed here separately
to highlight their importance for sustained programme success (see also section 1.3).

6.3.1 Capacity Development

Capacity development refers to contributions made by the Programme to the development

of essential CP and CP-related capacities in the host country. In this evaluation, four
target capacities were distinguished, respectively: resource productivity; environmental
management; entrepreneurship; and public-private partnerships (see also paragraph
4.5.5). A further distinction was made between three primary target groups for capacity
development, respectively: enterprises; the private sector (as represented by its industry
and business associations); and government (in host country).

The programme assessment in regard to capacity development is presented in Figure 6.5.

Considerable capacity development has been achieved covering mainly in the areas of
resource productivity and environmental management for most of the target beneficiaries,
leading to an overall positive rating on capacity development. The overall performance on
capacity development can be rated as satisfactory. The differences among the target
beneficiaries and target capacities are as followed justified.

¾Among the target beneficiaries identified for this programme evaluation, capacity
building has been highest for government, followed by enterprises and subsequently
private sector (the associations of employers, professionals etc). This is based on:

o Government: capacity development has been good in those countries where the
NCPC/NCPP succeeded in setting up an effective liaison with government, which
in some cases was helped by the fact that the NCPC was being hosted by a public
sector entity. Capacity development

154
Figure 6.5: Programme-level assessment on capacity development
Programme Scores: 1 = unsatisfactory, 2 = low, 3 = satisfactory, 4 = good and 5 = excellent

Programme Assessment: Capacity Development

4
programme scores

0
Resource Productivity Environmental Management Entrepreneurship Public-Private Partnership

enterprises private sector government

has been most profound at central environmental agencies, so that further

capacity development at local and regional levels and with national agencies with
economic, planning and technology mandates still needs to take place;

o Enterprises: capacity building has been good albeit limited to those enterprises
that participated in activities of the NCPC, in particular CP training and/or in-
plant demonstrations; and

o Private Sector: some capacity building in private sector entities (like industry
associations, chambers, etc.) did occur in particular in countries where NCPC is
hosted by a private sector entity.

¾Among the target capacities, capacity building was highest for resource productivity,
closely followed by environmental management. It was almost identical for
entrepreneurship and public – private partnerships, but capacity development on each
of these was markedly lower than on environmental management and resource
productivity. The detailed assessment result for each target capacity is as follows:

o Resource Productivity: capacity building is evident from the fact that the NCPCs
have been able to clearly articulate and deliver the message of environmental
improvement through productive investments, and as a result many of the CP
options implemented and/or considered for implementation displayed potential
for considerable cost savings resulting from reduced use of natural resources
(energy, water, materials);

o Environmental Management: the NCPCs have enabled companies to improve

their environmental performance, and facilitated the introduction of

155
 environmental management functions, even though CP implementation has
typically not achieved compliance with environmental laws and standards.
Moreover, they have provided assistance to strengthen environmental policy and
its enforcement;

o Entrepreneurship: capacity development in entrepreneurship has been dependent

on the NCPC establishing the link between CP on one hand and productivity and
quality management on the other hand. This aspect has not been favoured by the
predominant engineering approach taken to CP auditing and implementation.
Capacity development in entrepreneurship has therefore been limited to those
countries where the NCPC is hosted by an organisation with a traditional focus on
productivity and/or entrepreneurship; and

o Public-Private Partnerships: some capacity building occurred but only indirectly

as NCPCs did not explicitly focus on potential of CP to bridge traditional divides
between public and private sector on industrial environmental management and
resource use. Typically the NCPC is rather rooted in either the public or private
sector, with limited potential for achieving a public-private partnership.

6.3.2 Ownership

The second of the secondary evaluation criteria covers ownership. It addresses the
commitments and contributions by local stakeholders to advance the UNIDO-UNEP CP
Programme. Ownership is assessed in regard to the CP concept (as a business
improvement concept or practice), of the NCPC (as a local CP service delivery
institution) and of the global programme. Contributions are considered from enterprises
(individual businesses and other organisations), the private sector and government.

Figure 6.6 presents the programme level result of the assessment on ownership. Even
though this Figure displays a divergent picture among the Programme elements and
between the stakeholders, the overall level of ownership was generally rated low.

Figure 6.6: Programme-level assessment on ownership

Programme Scores: 1 = unsatisfactory, 2 = low, 3 = satisfactory, 4 = good and 5 = excellent
Programme Assessment: Ownership

4
programme score

0
CP Concept NCPC Institution International CP Programme

enterprises private sector government

156
The level of ownership of the CP concept and the NCPC institution is about equal, but
with government having the highest level of ownership for the NCPC institution and
enterprises the highest level of ownership of the CP concept. This reflects the fact that
government is most committed to maintain a national CP centre, while other stakeholders
view such Centre as a means for service delivery and not an end in its own right.
Enterprises, in particular those that have been assisted by the NCPC, are most committed
to CP as a business improvement tool and this has brought them direct productivity gains
and/or environmental benefits.

Apart from a weak commitment from national governments in host countries, there is no
ownership of the CP Programme. This is hardly surprising since the emphasis of
Programme implementation was on implementation at the country level and on
establishing NCPCs. Institutions other than the NCPCs did not benefit to a significant
extent from the Programme. An exception from this might be the efforts of UNEP to
promote its International Cleaner Production Declaration.

6.4 Summary Assessment

The previous sections presented the detailed programme evaluation individually for each
primary and secondary evaluation criteria. To wrap up the evaluation the contributing
scores within each evaluation criteria have been averaged, to arrive at a single rating on
each of the six evaluation criteria. The result thereof is shown in Figure 6.7. The variation
in the averaged programme level assessment scores for the six evaluation criteria is
relatively limited. Sustainability and relevance have the highest scores (respectively 3.0
and 2.9), followed by effectiveness, efficiency and capacity building (respectively 2.5, 2.5
and 2.4), and then followed by ownership (score of 1.3).

Figure 6.7: Averaged programme-level assessment for all evaluation criteria

Programme Scores: 1 = unsatisfactory, 2 = low, 3 = satisfactory, 4 = good and 5 = excellent
Programme Assessment

4
averaged programme scores

0
relevance effectiveness efficiency sustainability capacity ownership
development

157
Figure 6.7 shows that the programme assessments are in the range of being satisfactory.
Given the ambitions, complexity and scope of the UNIDO-UNEP CP Programme this
should be regarded as a good assessment result. It should also be kept in mind that the
evaluation framework was based on the programme documentation which defines the
programme in an over-ambitious way (as discussed in Chapters 2 and 5). As a result the
evaluation framework was also formulated broadly and thereby included elements that
were in the programme documentation but that had not been actively pursued by the
NCPCs/NCPPs and/or Programme management. This suppressed the evaluation scores,
for example by including entrepreneurship and public private partnerships as target
capacities, the score for capacity building decreased. Likewise consideration of
networking for all primary evaluation criteria, which has with a few exceptions not been
specifically funded, also decreased the assessment on all primary criteria.

Sustainability and relevance are thus, in principle, good and there are external factors that
are boosting the relevance of CP. The lower scores for effectiveness and efficiency show
that there is considerable potential for streamlining programme delivery and
administration and target it more profoundly towards national priorities and capabilities in
the different host countries. Doing so, has the potential to improve the Programme’s
performance with regard to the secondary evaluation criteria (capacity development and
ownership) and further bolster relevance and sustainability.

This programme assessment is underpinned by the following key findings.

1. CP is of continued and rising relevance.

CP is generally considered relevant by government, private sector and other
stakeholders in host countries for the UNIDO-UNEP CP Programme. Several current
global trends cause the relevance of CP to rise, but the presence and significance of
these trends varies greatly between the host countries. These include: worsening
industrial pollution situation and high industrial resource use (including energy, water
etc.); entering into force of MEAs; globalisation and trade liberalisation (including
free trade agreements); and pressure from international buyers and investors.

2. The UNIDO UNEP CP programme has produced valuable outputs and outcomes
in all 18 countries visited for an independent evaluation.
Its principal achievement has been in putting CP on the agenda of government and
business, building capacity for CP, development of information materials,
implementation of good housekeeping and low/intermediate technology options in
selected companies and policy change in some countries. The evaluation confirmed
that in all countries in which CP activities were started some CP activity is still
ongoing. In several countries the success of the NCPC was seriously compromised by
difficulties encountered in securing the commitments and meeting the expectations of
the host institutions. In some of these countries this issue was effectively addressed
through a re-formulation of the national implementation strategy for the NCPC. The
NCPCs that are no longer institutionally funded through the UNIDO-UNEP CP
Programme do not maintain close relations with the Programme and some no longer
have CP as their core service area.

3. The potential of the CP Programme has not been fully exploited.

The country visits demonstrated that each NCPC is unique in its institutional setting,
activities and achievements, with considerable differences from the ‘idealised’ NCPC
as being portrayed by the Programme and advocated by its management. The

158
 Programme has not yet demonstrated flexibility to adapt its support to the specific
needs and activities of the different countries and enable different types of NCPCs to
fulfil niche roles that are most appropriate and effective in their specific national
contexts.

The funding of the UNIDO-UNEP CP Programme has been mainly on a country-by-

country basis. There has been an assumption that the Programme would create
networking and professional development/learning opportunities, but no mechanism
was created to fund such programmatic activities. This absence of programme-based
funding has further contributed to a scattered approach to networking and learning,
with limited opportunities for capturing and advancing best practices and for
strengthening and managing the network.

4. The design and strategy for the CP Programme have major shortcomings.
There is no over-arching programme document. The Programme’s overall objectives
are therefore not always explicit and causing stakeholders’ expectations of the
Programme to vary. A logical means-end relationship between the overall objectives,
the impacts, outcomes and outputs, and activities of the Programme has not been
established, which has led to a rather standardised approach for the introduction of
CP on a project-by-project basis and to a lack of demand-based models for national
implementation of the Programme that customise to the unique national institutional
set up and capability portfolios of each of the Centres.

Some of the Programme objectives (including implicit ones (32)) have been far too
ambitious in light of available resources and project timelines, in particular: market
creation for CP services, decoupling of economic growth from worsening
environmental pollution and international market access for developing countries’
manufactured goods. While demand for CP services is on the rise in those countries
where substantive policy change has taken place, overall the Programme has yet to
contribute significantly to the emergence of markets for CP services in the NCPC
host countries. The contribution of the Programme to the decoupling of industrial
development from environmental pollution is also not measurable at sector, regional
or national levels. There is also no evidence of a contribution of the CP programme to
improved international market access of developing countries’ manufactured goods.

A number of strategy documents have been produced for the Programme over time
that expand the Programme to cover a broader set of topics, under the headings of CP
Plus, Sustainable Consumption and Production and/or Corporate Social
Responsibility. There is uncertainty, as these strategy documents appear to be used
for programme management and promotion, but have not been incorporated into
national project documents. Integration of new topics into NCPC service delivery at
the national level has therefore not yet materialised. The expansion of the
Programme’s scope at the global level has caused the Programme to drift from its
initial mission to achieve widespread uptake of CP in the host country. There is a
preference from many national stakeholders, often very strongly, albeit not
necessarily shared by the respective NCPC, to maintain a strict focus on CP (which
by definition includes energy efficiency, (hazardous) waste minimisation and
chemicals management), due to the urgency of the environmental health situation in
32
Implicit objectives are those found in documents of individual NCPCs (project documents) or in older documents that are
no longer valid (e.g. programme document for the set up of the first five NCPCs) or in documents developed by individual
NCPCs that received support from the programme.

159
 and around factories. In many countries there are also other institutions that might be
better positioned to advocate the emerging topics of CSR, SCP and CP+.

5. Weak monitoring and reporting limits adaptive and effective programme

management.
Reporting of Programme achievements is generally insufficient to monitor outcomes
and impacts against Programme objectives, which hinders adaptive management and
continuous improvements in service delivery, at national and programme levels. The
set of programme indicators used for annual reporting is aggregated from national
reports. These national reports contain outputs from training and in-plant
demonstrations, estimates of impacts of CP implementation on basis of assessment
reports, and financial independence data for the NCPC. Standardisation of data
collection from different NCPCs/NCPPs remains weak, while also no data are
collected for NCPCs that do no longer receive institutional funding through UNIDO.
Absence of indicators for capacity building, including policy change, market
development, awareness creation and technological capability, is of further concern.
Moreover post-implementation measurement of benefits achieved from in plant
demonstrations/CP assessments to produce ‘success stories’ is not routinely taking
place.

6. The ‘win-win’ premise of CP is true only under specific circumstances.

The ‘win-win’ premise on which the UNIDO-UNEP CP Programme is largely based
is not universally achievable in the host countries for the Programme due to lack of an
enabling framework (including environmental policy/enforcement and resource
pricing). The continued use of the ‘win-win’ premise has created expectations among
national stakeholders (in particular in the private sector) that cannot be met and in
turn weakened their buy-in into the Programme.

7. The CP Programme was not very successful in EST Transfer.

Some CP technology investments have been facilitated through the Programme, often
by utilising available green credit lines (for international technology acquisitions)
and/or deployment of local engineering design and fabrication capacities (for
upgrading of local technologies). Overall however the Programme has made little
headway in transferring ESTs, neither through the regular activities of the NCPCs nor
through specific CP technology transfer initiatives. As this recognised CP potential
therefore remains largely untapped, there is an urgent need to review best practices in
technology transfer, adaptation and replication, and redesign Programme activities
accordingly.

8. Creation of NCPCs/NCPCs is an appropriate way for capacity building in CP but

attention for their institutionalisation has been limited.
The UNIDO-UNEP CP Programme has defined NCPCs by their portfolio of
standardised CP services. The institutional dimension of the NCPCs (e.g. the NCPC’s
role vis-à-vis other types of institutions, the NCPC’s role in the national innovation
system) has therefore not been sufficiently considered in many cases. At the
programme level this is evident from the absence of explicit institutional objectives
for the NCPCs and also from the lack of a clear strategy for ongoing engagement with
NCPCs that no longer receive institutional funding from the UNIDO-UNEP CP
Programme. At the national level this is evident from unresolved legal status and/or
compromised independence of many NCPCs, and limited accountability and
transparency of the NCPCs to local stakeholders representing the public and private

160
 sectors. The fact that no specific analysis was performed of the national context
(economic, environmental and institutional) has contributed to this shortcoming.

9. The potential for cooperation with other initiatives has not been exploited.
The evaluation found only limited evidence of ongoing collaborations within the UN
agencies and with other UN Agencies, with donors other than the ‘current’ UNIDO-
UNEP CP Programme donors, and with other initiatives in the field of industry,
environment and sustainability. Given the multitude of such initiatives, there is an
unexploited potential to leverage expertise and resources at the programme and
national levels. The evaluation found that relevant areas of collaboration are: (i)
between UNIDO, UNEP and other UN agencies (e.g. UNDP, ILO, FAO); (ii) with
current programme donors (in particular Austria, Switzerland, Italy, Norway) and
other donors with similar CP initiatives outside, or in competition with, the UNIDO-
UNEP CP Programme (e.g. GTZ, DANIDA, SIDA, USAID, EU); (iii) with private
sector initiatives (e.g. WBCSD, APO); and (iv) with professional initiatives (in
particular the Regional Roundtables for Sustainable Consumption and Production).

High expectations exist for networking among NCPCs and possibly with similar CP
centres and projects currently outside the UNIDO-UNEP CP Programme. The
absence of specific objectives, outputs and outcomes for networking, made the
assessment of the Programme’s networking achievements difficult. Some networking
is achieved through collaborative projects and regional networking initiatives, and
outside of the UNIDO-UNEP CP programme through the system of regional
roundtables for sustainable consumption and production. There is hardly any
interaction between the Programme management and the NCPCs once the direct
funding relation through UNIDO has ceased, leading to distancing of these NCPCs
from the Programme. Networking needs and opportunities of the NCPCs and the
UNIDO-UNEP capabilities to meet those have not yet been sufficiently assessed. The
same is true for the intended role of a network vis-à-vis other networking initiatives at
the global (e.g. regional SCP roundtables, WBSCD) or regional (e.g. LatinNet, GTZ
networks) levels.

10. The valuable contribution of the programme to national capacity building is not
sufficiently communicated.
UNIDO, UNEP and Donors have a tendency to present all NCPCs equally as ‘their’
institutions (33), regardless of their national ownership and governance structures,
substantially different activity portfolios and funding models. The currently presented
view that NCPCs can be directed by UN agencies and Donors to promote UN and
Donor goals needs to be replaced by the notion that NCPCs, in particular those that
are no longer institutionally funded through the UNIDO-UNEP CP Programme, are
partners of the UN in promoting CP and sustainable industrial development in their
home countries.

11. There is a trade-off between financial independence and sustained impact.

The evaluation showed that the sustainability of the Programme’s achievements in
building CP capacity, implementing CP in companies and CP-promoting policies is

33
Operationalising UNIDO Corporate Strategy, p. 66 “the Organization will continue to develop the technical cooperation
services offered through its worldwide network of National Cleaner Production Centres (NCPCs) and National Cleaner
Production Programmes (NCPPs).”; or page 83: “The cleaner and sustainable production (CP) strategy of UNIDO aims at
utilizing the National Cleaner Production Centres (NCPCs) to implement the following two specific sets of
interventions:…..”

161
 generally high. It is however noted that the priority assigned to financial sustainability
of the NCPC as a national institution (largely through income from services) can
become counterproductive to achieving sustained effects and impacts as measured by
programme objectives. An example can be found in NCPCs that focus their service
delivery to larger businesses (including subsidiaries of trans-national corporations)
that can pay for services (but may not be target groups for the donor agencies or
illustrative examples for local CP potential) and, in turn, terminate service areas that
are of public interest (e.g. promote compliance through voluntary agreements) and
may limit training in order to avoid enabling competitors to enter the market.

162
 Part III:

Conclusions &

Recommendations

163
7
Conclusions and Recommendations
_______________________________________

7.1 Main Conclusions

This chapter presents the conclusions and recommendations from this programme
evaluation of the UNIDO-UNEP CP Programme. In its assessment of the Programme, the
evaluation team found that relevance and sustainability of the Programme are good, with
scope for improvement particularly for effectiveness and efficiency, which could result in
better targeted, customised and streamlined interventions at the national level, which in
turn could further bolster relevance and sustainability, as well capacity development and
ownership. The conclusions build upon the summary assessment (presented in section
6.4) and integrate the results from portfolio analysis (presented in Chapter 5) and
programme level assessment (presented in sections 6.2 (primary evaluation criteria) and
6.3 (secondary evaluation criteria)).

The conclusions and recommendations are organised in twelve clusters. Each cluster
provides a set of interrelated opportunities for improving aspects of the design, operation
and management of the UNIDO-UNEP CP Programme. These clusters are:

1. Relevance: potential of CP to contribute to national socio-economic and

environmental priorities in the host countries;

2. Impact: results of the UNIDO-UNEP CP Programme in particular the uptake of CP

concepts, practices, technologies and policies in the host countries;

3. Design & Strategy: means-ends relationship between objectives, impacts, outcomes

and outputs, and objectives of the UNIDO-UNEP CP Programme;

4. Focus (or contents): CP and related concepts that are being promoted through the
UNIDO-UNEP CP Programme;

5. Networking: cooperation, information exchange and collective learning among and

between the NCPCs/NCPPs;

6. Funding Model: types of funding available to the UNIDO-UNEP CP Programme and

the mechanisms for its allocation and distribution to activities of the NCPCs/NCPPs;

7. Centre Model: institutionalisation and positioning of NCPCs/NCPPs into nationally-

directed and/or locally–owned service providers;

8. NCPC Services: types of services delivered with support from the UNIDO-UNEP CP
Programme through the NCPCs/NCPPs;

165
9. Management & Monitoring: arrangements in place for the daily operation of the CP
Programme, and monitoring of its achievements against expected outputs, outcomes,
impacts and objectives;

10. Administration: provisions made to manage contracts and disburse programme

funding according to internationally acceptable accounting standards;

11. Governance & Ownership: accountability and transparency in decision making at

programme and national levels; and

12. Excellence: ambition of the CP Programme to play a leading role in international

efforts to foster the uptake of CP, deliver best practice CP services and establish
NCPCs/NCPPs as centres of excellence.

The main recommendations of these clusters provide an integrated framework for

developing and managing the UNIDO-UNEP CP Programme for growth, impact and
maturity of the NCPCs/NCPPs. The twelve clusters with their main conclusion,
supporting evidence and overarching recommendation are provided in Table 7.1. These
are explained in detail and complemented with supportive conclusions and
recommendations in Section 7.2. Next, Section 7.3 contains some final remarks on this
programme evaluation.

166
Table 7.1: Overview of main conclusions and overarching recommendations
Cluster Main Conclusion Key Evidence Reference Overarching Recommendation
1. Relevance CP is relevant and its relevance is on the x Businesses and other x Country reports and The CP Programme should be continued to
rise due to worsening industrial organisations have been analysis of results in assist developing and transition economies
pollution, resource scarcity, entering into able to benefit from Paragraph 4.4. to develop capacity to apply CP practices,
force of MEAs, trade liberalisation and implementation of CP. technologies, methodologies and policies in
globalisation, buyer pressure and greater x Several countries have support of their national socio-economic and
government and community awareness. introduced CP policies environmental priorities.
and strategies.
2. Impact The Programme was successful in x Feed back received from x Self evaluation The NCPCs/NCPPs should capitalise on
establishing CP initiatives in each host all NCPCs/NCPPs for results, covered in their achievements and target their service
country and all were reported to be the self evaluation country profiles and delivery better to increase impact of their
active. For the visited countries it could x Demonstrable results analysed in Chapter services on the uptake of CP practices,
be confirmed that the NCPC had from service delivery by 3. technologies and policies, in particular
produced valuable outputs and outcomes NCPC in all visited x Country reports and during the phase of support through
in particular with regard to awareness countries. analysis of results in UNIDO-UNEP and donors.
raising, training, implementation of low Paragraph 4.4.
and intermediate technology CP options
and, in some countries, policy change
3. Design & There is no programme document x Absence of programme x Review of The Programme should be guided by a
Strategy covering the overall objectives, the document, and programme succinct programme document, with a clear
strategy and intervention logic and the discrepancy between documents (Chapter strategy, a justification of the intervention
different expected contributions from revised programme 2). logic and the specific roles and contributions
UNIDO, UNEP and local stakeholders. strategies and national x Diversity of national from UNIDO, UNEP and local and
Existing strategy documents are not projects that control implementation international stakeholders.
useful for Programme management. programme strategies is not
implementation. being captured at
programme level
(Chapter 5).
167

167
 168
Cluster Main Conclusion Key Evidence Reference Overarching Recommendation
4. Focus The expansion of the scope of the CP x Absence of framework x Review of The Programme should re-establish its
(Contents) concept that has gradually occurred in that explains new programme primary focus on CP and articulate a dual
the Programme over time catalysed by elements and connects documents (Chapter strategy for its further development to
interests of the donors and the UN these to the core CP 2). enable specialisation (in policy and/or
agencies, is not widely understood by all concept. x Self evaluation technology) and diversification (socially
programme stakeholders and lacks x Limited awareness and survey (section 3.5). driven and/or environmentally driven) of
widespread endorsement by the interest from x Country reports and NCPCs/NCPPs as they and their national
NCPCs/NCPPs and their national NCPCs/NCPPs in new their comparative stakeholders see fit in their respective
stakeholders. topics. analysis (Chapter 4). national contexts.
x Feed back from x Portfolio analysis
interviews with (Paragraph 5.3.2)
government, private
sector and other
stakeholders in visited
countries.
5. Networking The Programme has not formulated a x Absence of a networking x Programme review The Programme should formulate a clear
distinct strategy with tangible objectives, strategy and dedicated (Chapter 2) networking strategy with tangible and
outcomes and outputs for networking funding for networking. x Country reports and realistic outcomes, outputs and activities,
among NCPCs and the resource needs x High expectations for comparative analysis which could be realised by supporting a
for its facilitation and technical support networking encountered of national membership based network that would be
through the UNIDO-UNEP Programme in visited countries, but evaluation results on open to qualifying institutions, including
management have not been identified. not being met due to low relevance, NCPCs established by the UNIDO-UNEP
networking intensity. effectiveness and CP Programme as well as eligible other CP
efficiency of service providers
networking (in
Section 4.4)
6. Funding Model The predominant model for funding of x Absence of dedicated x Programme review The Programme should adopt a dual funding
the Programme as a collection of country funding for networking (Chapter 2). model at Programme and national levels: (1)
projects has hindered effective and other programmatic x Independent country country-based block funding to support
networking and constrained the multi-country projects. evaluations for NCPCs in their establishment phase; and (2)
Programme in developing and delivering x Positive experience with participating programme funding for (i) competitive
specialist services on a multi-country multi-country projects, countries (e.g. India grants to multiple eligible NCPCs and
basis. e.g. GERIAP and on CP and Vietnam). possibly qualifying other CP service
and MEAs. providers for project based specialisation
and/or diversification; and (ii) networking
initiatives.

168
Cluster Main Conclusion Key Evidence Reference Overarching Recommendation
7. Centre Model The capacity building model through x Neither documents at x Programme review The Programme should articulate
NCPCs/NCPCs is relevant, even though Programme level nor (Chapter 2). institutional objectives and scenarios for a
the Programme defines NCPCs by their project documents do NCPC so that institutionalisation of the
service categories without providing address institutional NCPC can be monitored and provisions be
clear institutional perspective(s) for the aspects of NCPC created to accommodate both the public
NCPC, both during and beyond their establishment. interest and private benefit functions of the
phase of institutional funding through NCPC services over time.
the UNIDO-UNEP CP Programme.
8. NCPC Services The Programme has outlived its initial x New service areas have x Programme review The Programme should support the
design of services which was based on a been added to the (Chapter 2), analysis NCPCs/NCPPs to undertake periodic
standard package of NCPC services to Programme (see also of self evaluation assessments of the national status of CP, to
be delivered through one single national under focus). results (Chapter 3) define and review their strategic niche with
centre, as countries that have built CP x Other CP and CP-related and independent service portfolios that are most appropriate
capacity in different institutions require service providers exist, evaluations (Chapter and effective in their respective national
more tailor made NCPC services. and there is an 4). contexts.
expectation that these will
be serviced by the
NCPC/NCPP.
9. Management & Reporting on Programme achievements x Irregular progress reports x Programme review The Programme should adopt a results--
Monitoring is generally insufficient to assess at programme level and (Chapter 2) and based management model at Programme and
outcomes and impacts against annual reports only for analysis of results at national levels and develop a comprehensive
Programme objectives which prevents funded NCPCs/NCPPs. the national level in system to monitor performance in capacity
adaptive management and continuous x Low evaluation scores on visited countries and building, institutional development and
improvement of the Programme’s effectiveness and comparative analysis results and impacts from CP service
performance. efficiency of programme thereof in Section 4.4 delivery. It should also monitor that agreed
management in visited project structures, governance arrangements
countries. and contributions from host countries and
institutions are being achieved.
10. Administration The UNIDO CP Unit and x Low evaluation scores on x Reports of the The Programme management should
NCPCs/NCPPs have ultimately been effectiveness and independent streamline programme administration and
able to meet administrative efficiency of programme evaluations for 18 shift to the extent feasible financial
requirements, including financial administration in visited countries and responsibility and accountability to the
administration and contracts’ countries. summary of national NCPCs/NCPPs and/or national stakeholders.
management and disbursement of funds, evaluation results in
but repeatedly not in a timely manner. section 4.5)
169

169
 170
Cluster Main Conclusion Key Evidence Reference Overarching Recommendation
11. Governance & The Programme has not established a x Absence of governing x Programme review The Programme and the NCPCs should
Ownership transparent and accountable governance board at Programme level (Chapter 2). adopt transparent and accountable
structure for gathering feed back from and at centre level in x Self evaluation governance structures at Programme and
stakeholders, beneficiaries and NCPCs many countries (in information on board national levels, preferably with small boards
into its strategic planning and ensuring particular for NCPCs no membership with participation of private sector,
adequate oversight over implementation longer institutionally (contained in country government and civil society, that assume
of the Programme. The governance of funded through the profiles). accountability for the success of the
NCPCs is of varying effectiveness, Programme). x Independent Programme and the NCPCs.
accountability and transparency. x Existing national boards evaluations for 18
tend to be top heavy and visited countries
decision making (analysed in Chapter
procedures are not always 4, in particular
clear. Section 4.3).
x No provisions for
ongoing engagement with
NCPCs no longer
institutionally funded
through the Programme.
12. Excellence Despite its ambition for excellence, x Programme management x Programme review The Programme should establish a culture of
thematic leadership in the Programme is not resourced to (Chapter 2) and experimentation and continuous
management is weak, as well as its undertake effective peer independent improvement in CP service delivery.
incentives and opportunities for realising review and/or quality evaluations (Chapter Sufficient programme funding should be
continuous improvements in control on services of 4). made available for that purpose.
development, adaptation and replication NCPCs/NCPPs and of the
of CP services and initiatives. Programme’s
international consultants.
x Programme relies for its
thematic inputs on a
narrow base of
international consultants
with highly comparable
competencies

170
7.2 Detailed Conclusions and Recommendations

This section expands on the main conclusions and recommendations as summarised in

Table 7.1. Each cluster is addressed consecutively.

7.2.1 Relevance

Relevance is already one of the key strengths of the UNIDO-UNEP CP Programme as the
potential of CP to improve productivity and environmental performance is valuable in
light of environmental, economic, trade and technology policies, as well as important for
businesses to remain and/or become competitive. This is an important justification for
continuation of the Programme. The independent evaluations in the 18 visited countries
revealed that in several countries the NCPC serves primarily the manufacturing sector,
even though other sectors (e.g. rural development, agro- and forestry industries, fisheries,
tourism, services and/or mining) are far more important in the country’s economy and for
achieving its socio-economic development objectives. Likewise, it was found that CP
continues to be approached as an environment and resource productivity strategy, thereby
ignoring the opportunity to use CP as a practical tool to foster entrepreneurship, enterprise
development and public-private sector cooperation. In moving the Programme forward, it
is therefore recommended that an effort is made to make CP more relevant for the
specific development and environmental context of the respective host country, by
exploiting policy synergies, customising CP concepts and methods, and targeting of CP
service delivery to national priority sectors. The detailed conclusions and
recommendations are provided in Table 7.2.

171
Table 7.2: Detailed conclusions and recommendations on relevance
Cluster 1. Relevance
Conclusion Recommendation
CP is relevant and its relevance is on the The CP Programme should be continued to
rise due to worsening industrial pollution, assist developing and transition economies to
resource scarcity, entering into force of develop capacity to apply CP practices,
MEAs, trade liberalisation and technologies, methodologies and policies in
globalisation, buyer pressure and greater support of their national socio-economic and
government and community awareness. environmental priorities.
Contributing Conclusions Supportive Recommendations
1.1 CP has been implemented as an 1.1 The Programme should place greater
environmental and resource productivity emphasis on the synergistic potential of CP
strategy, with limited focus on its to strengthen the private sector, as a driver
potential to contribute to enterprise for socio-economic development, including
development, regional development and in rural areas.
strengthening of public-private 1.2 The Programme should adopt a more
partnerships. inclusive approach to CP implementation in
1.2 Apart from initiatives in some countries to host countries and support NCPCs/NCPCs
foster CP implementation in service more effectively in developing CP concepts,
organisations (particularly hotels and methods and policies that are customised to
government offices), the Programme has the needs and opportunities of those sectors
had a relatively narrow focus on the that are considered most important for the
manufacturing sector and opportunities to national economy and/or environmental
customise CP for application in other improvement. This could be enhanced
sectors like agriculture, fisheries, mining, through cooperation with other agencies
construction, etc., have not been (e.g. FAO).
systematically pursued despite the 1.3 The Programme should formulate explicit
importance of these sectors in the national criteria and/or auditable protocols for
economies of many of the host countries. prioritising among potential target groups
1.3 Prioritisation of industry sectors and for service delivery by the NCPC/NCPP to
business sizes as target groups is maximise potential benefits for national
opportunistic, both at national and development goals and environmental
programme levels, and poorly justified by priorities, particularly during the phase that
perceived potential for CP implementation the NCPC/NCPP is institutionally funded
and its estimated environmental and through the Programme.
productivity benefits.
1.4 Many NCPCs have focused service
delivery on medium to larger businesses,
typically with international ownership
and/or customers, as these supposedly
have a greater capacity to pay for NCPC
services.

7.2.2 Impact

A principal impact at Programme level is that the NCPCs/NCPPs that were established
over the duration of the Programme all reported to remain active in some form in CP
promotion and/or implementation (34). At the national level, the independent evaluations
of the NCPCs in the 18 visited countries confirmed impact had been achieved
predominantly through implementation of low and intermediate technology CP options,
training and awareness creation. An effort is urgently needed to identify opportunities to
improve the impact of the Programme, and incorporate these in a logical programme
34
All expect one of the NCPCs/NCPPs provided some information to the evaluation team. In case of Ethiopia, the
evaluation team had to rely on reports from the UNIDO CP Unit in regard to ongoing activity.

172
document (see also recommendations in paragraph 7.2.3). At programme level, this might
be achievable by putting more emphasis on capturing and disseminating international best
practices for market-led CP promotion and implementation (also conducive to excellence
as covered in paragraph 7.2.12), and providing guidance on integrated approaches for
creating demand and supply for CP services. Improving networking and information
sharing is an important mechanism for this (as addressed separately in paragraph 7.2.5).
At the national level, impact can be improved by better targeting of activities, in
coordination with other CP and related initiatives in the host country and through
deploying international best practices in planning and delivery of CP services. Table 7.3
provides a comprehensive summary of the detailed conclusions and recommendations in
regard to impact.

Table 7.3: Detailed conclusions and recommendations on impact

Cluster 2. Impact
Conclusion Recommendation
The Programme was successful in establishing CP The NCPCs/NCPPs should capitalise on their
initiatives in each host country and all were achievements and target their service delivery
reported to be active. For the visited countries it better to increase impact of their services on the
could be confirmed that the NCPC had produced uptake of CP practices, technologies and
valuable outputs and outcomes in particular with policies, in particular during the phase of
regard to awareness raising, training, support through UNIDO-UNEP and donors.
implementation of low and intermediate technology
CP options and, in some countries, policy change.
Contributing Conclusions Supportive Recommendations
2.1 The Programme has not yet succeeded to identify 2.1 The Programme should adopt as one of its
best practices of CP service delivery from within explicit immediate objectives to capture,
and outside the UNIDO-UNEP CP programme develop and disseminate best practices in
and disseminate these effectively among NCPCs. market-lead CP promotion and
2.2 In several countries the NCPC is only one of the implementation. This would align the
actors involved in promoting the uptake of CP. programme better with the ‘global forum'
2.3 Even though demand for CP services is on the rise mandate of the UN agencies involved.
in those countries where substantial policy change 2.2 The NCPCs should monitor other
has been achieved, overall the programme has not developments promoting CP in their country
achieved to contribute to the development of and customise their service delivery to ensure
national markets for CP services that could sustain these complement and reinforce other
the operation of the NCPC. activities.
2.4 Planning and management of service delivery at 2.3 The Programme objectives (and national
the national level is often un-targeted which centre strategies) should reflect a
compromises its effectiveness and overall impact. comprehensive approach to demand creation
2.5 With limited resources the NCPCs face a trade off for CP services, through policy change,
between supporting a greater number of businesses environmental compliance, investment
with implementation of basic low/intermediate CP promotion and public awareness, and supply
options or assisting a smaller number of creation for CP services, through capacity
enterprises with identification and implementation building and training.
of high technology CP options. 2.4 The NCPCs should make a strategic choice in
their resource allocation to the demands for
basic CP practices and for specialised CP
technology services. The Programme and
NCPCs/NCPPs should proactively strengthen
collaboration with other national institutions
on meeting both demands.

173
7.2.3 Design & Strategy

The documents’ review (in Chapter 2) revealed that the Programme is being implemented
by default as a set of similar and partially connected national projects, instead of being
driven by a clearly developed and articulated programme document. The self evaluation
and independent evaluations (Chapters 3 and 4) confirmed that the absence of an
overarching programme strategy has dispersed the programme’s resources instead of
focused these around key objectives and a logical sequence of output, outcomes and
impacts. It is therefore strongly recommended that a Programme Document be developed
for the UNIDO-UNEP CP Programme as a matter of absolute urgency. Such programme
document should describe and justify the intervention logic, provide specific objectives
and outputs, outcomes and impacts at programme level, distinguish these from the
objectives and outputs, outcomes and impacts of CP implementation at the national level,
and provide a logical means-end relationship between these two levels. Indicators should
also be developed and interactions with other local and international initiatives should
also be dully considered. Table 7.4 provides a comprehensive overview of the detailed
conclusions and recommendations with regard to strategy.

174
Table 7.4: Detailed conclusions and recommendations on strategy
Cluster 3. Strategy
Conclusion Recommendation
There is no programme document The Programme should be guided by a
covering the overall objectives, the succinct programme document, with a clear
strategy and intervention logic and the strategy, a justification of the intervention logic
different expected contributions from and the specific roles and contributions from
UNIDO, UNEP and local stakeholders. UNIDO, UNEP and local and international
Existing strategy documents are not useful stakeholders.
for Programme management.
Contributing Conclusions Supportive Recommendations
3.1 The Programme’s overall intent to 3.1 The Programme should adopt clear, focused
decouple economic growth from and specific development objectives and
environmental degradation is widely expected outcomes related to decoupling
supported. economic growth from environmental
3.2 The Programme has been overburdened degradation through the greater uptake of CP
by promoting possible positive spin offs practices and technologies.
from CP into Programme-level 3.2 The Programme should make a clearer
objectives (e.g. gender equality, poverty distinction between contribution and
reduction). attribution with regard to its intended
3.3 Some of the expected outcomes (like the development objectives and impacts,
one to decouple economic growth from outcomes and outputs. Furthermore, some of
resource consumption and environmental the possible spin offs from CP could be
degradation) are over-ambitious, given turned into ‘conditions for implementation’
the relatively small size and catalytic rather than objectives (for example,
nature of the Programme. implementation to be neutral or positive with
3.4 For most of the interventions, regard to gender equality, community health,
effectiveness depends to a large extent poverty reduction, etc.)
on the development of other international 3.3 The design of the Programme strategy should
and local initiatives. be improved so as to establish a logical
3.5 The Programme’s focus is on volume of means-end relationship between development
service delivery without sufficient objectives, impacts, outcomes, outputs and
consideration for quality or impact of activities, including the proper definition of
such service delivery (e.g. increased indicators for: capacity building; CP
implementation of CP by businesses and implementation; policy change and creation
other organisations or policy change). of an enabling environment; market
3.6 Frequently, outputs are being used as development; and technology transfer,
substitutes for outcomes (e.g. projected adaptation and replication (including
savings from a CP assessment instead of investment).
actual post implementation benefits).
3.7 Not all of the expected outcomes of the
programme have been made explicit (e.g.
CP market creation), and some of the
explicit outcomes are not clearly linked
to the activities and outputs by a means-
end relationship.

7.2.4 Focus

Focus (or alternatively contents) refers to the set of main topics and concepts for which
the UNIDO-UNEP CP Programme is establishing capacity in the host countries. It was
concluded that the CP concept has been extended gradually over time to cover a broader
set of CP and CP-related topics, a process which was initiated by donors (in particular for
EST transfer and financing, and CSR) and the UN agencies (CP Plus, chemical leasing,
SCP). These extensions have been ‘added on’ instead of ‘integrated into’ the existing

175
Programme. Their interrelatedness and connection to core CP concepts has not been
properly established, leading to a degree of misunderstanding and ambiguity about the
evolving focus of the Programme. The portfolio analysis provided a suggestion to clarify
the focus of the Programme by distinguishing between diversified CP services and
specialised CP services (see section 5.3.3). There is also concern that the programme
additions will dilute or disperse the CP capacities built so far, whilst the task of achieving
widespread implementation of CP remains to be accomplished. It is therefore
recommended that the primary focus on CP is re-established, and that a framework be
provided to explain the interrelatedness of new elements and their connections with the
core CP concepts and practices. Doing so will also assist in defining service packages the
Programme can offer to NCPCs/NCPPs and potentially to similar CP Centres currently
not yet part of the Programme
Table 7.5: Detailed conclusions and recommendations on focus
Cluster 4. Focus (Contents)
Conclusion Recommendation
The expansion of the scope of the CP concept The Programme should re-establish its
that has gradually occurred in the primary focus on CP and articulate a dual
Programme over time catalysed by interests strategy for its further development to
of the donors and the UN agencies, is not enable specialisation (in policy and/or
widely understood by all programme technology) and diversification (socially
stakeholders and lacks widespread driven and/or environmentally driven) of
endorsement by the NCPCs/NCPPs and their NCPCs/NCPPs as they and their national
national stakeholders. stakeholders see fit in their respective
national contexts.
Contributing Conclusions Supportive Recommendations
4.1 Even though progress has been made in 4.1 The Programme should maintain a clear
putting CP on the agenda, a continued focus on CP to ensure that CP capacities
focus on CP will be needed to avoid built so far are being maintained,
deterioration of CP capabilities built and strengthened and utilised for achieving
achieve wider-spread uptake of CP wider-spread uptake of CP, including
practices and technologies. higher-technology opportunities.
4.2 New elements have been added to the 4.2 The Programme should provide an
Programme by the Programme integrative framework that logically
Management, and all of these are presented connects its focal areas.
equally as new components without clear 4.3 It is strongly suggested to use
terminology or an integrative framework to specialisation and diversification as the
explain their inter-relatedness and basis for formulation of the integrative
synergies with the core CP concepts. framework.
4.3 Some of the new elements introduced in the 4.4 The Programme should then articulate a
Programme are ‘specialisations’ that dual strategy for its further development to
improve the rigour and depth of service enable both specialisation and
delivery related to uptake of CP, with diversification of NCPCs, depending on
policy-intensive services and/or their national contexts.
technology-intensive capacities and
services.
4.4 Other new elements introduced in the
Programme are ‘diversifications’ that
broaden the scope of service delivery,
towards inclusion of social aspects (leading
to an expansion into Corporate Social
Responsibility) and/or inclusion of other
environmental aspects (leading to an
expansion into Sustainable Consumption
and Production).

176
4.5 The absence of a clear distinction between
specialisation and diversification has
further compromised the programme’s
effectiveness.
4.6 The size and diversity of the national
economy and the severity of industrial
pollution determine to a large extent
whether and how the NCPC can specialise
further in CP service delivery. Possible
areas of specialisation are technology
assessment and transfer, technical standard
setting, research and innovation,
investment advice, policy change,
curriculum development etc. Alternatively
such specialisation could be catered for at
the regional level.
4.7 Several NCPCs have opportunistically
embraced the opportunity for
diversification, but support for this from
national stakeholders is limited to those
countries where industrial pollution is not
yet an overarching national priority and/or
where the current size of the NCPC (and
contribution from the programme) is
relatively large compared to the total size
of the national economy.

177
(see also under networking, in paragraph 7.2.5). It could also support clarification of the
roles and contributions of UN agencies and donors involved, while also enabling
cooperation with other agencies and stakeholders not yet involved. Table 7.5 contains the
detailed conclusions and recommendations in full.

7.2.5 Networking

With the expansion of the geographic coverage of the UNIDO-UNEP CP Programme to

some 35 countries, the Programme has a legitimate claim to have created one of the
largest developing countries’ based networks of CP practitioners. The networking
expectations from NCPCs/NCPPs and their national public and private sector
stakeholders with regard to networking are high, albeit in most cases non-specific.
Programme management has been considering strengthening of the network for some
time, but the current networking intensity remains still low resulting in expectations
remaining unmet. The positive exceptions are project based networking in multi-country
projects (e.g. GERIAP) and regional networking initiatives (in particular LatinNet). No
overarching networking strategy has been defined for the Programme, and no dedicated
funds are available on an ongoing basis. The challenge therefore remains to turn the set of
national centres created by the Programme into a developing countries’ lead network of
service providers with different capability- and service-profiles and ownership and
funding structures, but united by a shared commitment to foster the uptake of CP
concepts, practices, technologies and policies, beyond their private commercial interest to
sell CP-related goods and services. The primary aim of the network should be to capture
from, and advance within, the network best practice methods, policies and technologies
for implementation of CP. Table 7.6 provides a comprehensive set of detailed conclusions
and recommendations for networking.

It is recommended that a networking strategy be developed and implemented in

consultation with (representatives of) currently funded and previously funded NCPCs and
possibly some CP centres not established through the Programme. The strategy should
define activities, outputs and outcomes, and roles and responsibilities for network support
and facilitation and network members, as networking will only be effective with an
ongoing effort from all participants to stay up-to-date and useful for members. To ensure
sustainability, the network will have to be driven by the CP centres themselves, with the
UNIDO-UNEP CP Programme playing a facilitating role in its establishment. A key
consideration will be to establish criteria for accessing the different networking services.
The current default criterion of being established through the UNIDO-UNEP CP
Programme is becoming gradually outdated and excludes a-priori the valuable activities
and outcomes of other CP service providers not established through the Programme. As
argued in Section 5.5 it is suggested that the network be established as a membership-
based association of CP service providers with statutes and eligibility criteria, rights and
obligations for different categories of membership. Doing so will be transparent and put
the burden of proof to NCPCs/NCPPs and others wishing to become member of the
network. The different categories of membership can then also be used to deliver different
packages of diversified and specialised CP services (as discussed under focus in
paragraph 7.2.4) and manage eligibility for competitive grant funding from programmatic
resources for topical multi-country projects (see also paragraph 7.2.6 regarding funding
model).

178
Table 7.6: Detailed conclusions and recommendations on networking
Cluster 5. Networking
Conclusion Recommendation
The Programme has not formulated a The Programme should formulate a clear
distinct strategy with tangible networking strategy with tangible and realistic
objectives, outcomes and outputs for outcomes, outputs and activities, which could be
networking among NCPCs and the realised by supporting a membership based
resource needs for its facilitation and network that would be open to qualifying
technical support through the UNIDO- institutions, including NCPCs established by the
UNEP Programme management have UNIDO-UNEP CP Programme as well as eligible
not been identified. other CP service providers.
Contributing Conclusions Supportive Recommendations
5.1 There are high expectations among 5.1 The Programme management should in
the NCPCs and their national consultation with the NCPCs/NCPPs define a
stakeholders for accessing CP dual support strategy based on (1) management
technology information and sharing and administration of project funding for those
of best practice methods, tools and NCPCs/NCPPs that receive institutional (or
policies for, and related to, CP block) funding through the Programme; and (ii)
through the UNIDO-UNEP CP provision of expertise and networking
programme. opportunities to all NCPCs/NCPPs and other
5.2 Apart from regional networking similar CP service providers.
initiatives and multi-country 5.2 The networking component should aim to capture
projects, the networking intensity in from, and advance within, the network best
the current Programme is too low to practices in promotion and implementation of
be effective or efficient, and concepts, technologies and policies for, or related
networking expectations are to, CP, for example through task forces,
therefore generally not met. conferences, study tours, joint thematic projects
5.3 Even though in several countries the and exchange of personnel and information
NCPC is not the only institution between network members.
advocating CP or associated 5.3 Such a strategy could be based on supporting the
concepts and practices, publicly creation of a membership-based association of CP
and/or privately funded, the institutions (not only formerly or currently funded
Programme does not achieve NCPCs/NCPPs), with clear statutes with
effective engagement or eligibility criteria and obligations for
collaboration with such other membership, but also clear benefits and services
institutions, neither at the national for members.
nor at the Programme levels. 5.4 In case network management is being established
as one of the Programme’s core functions, careful
consideration has to be given that appropriate
resources are devoted to that end, preferably on a
programmatic and at least medium term basis.

7.2.6 Funding Model

The funding model applies to the way the Programme as well as the NCPCs/NCPPs are
funded. Currently the Programme is almost exclusively funded on a country-by-country
project basis, creating very limited opportunity for multi-country initiatives, including
networking and specific projects. Funding to the NCPCs is provided as a block grant
(against eligible expenditure), and in principle only for an establishment period (even
though in practice this has been extended once or twice for several NCPCs). Catalysed by
donor-interests, the Programme management has been very much focused on achieving
financial independence of the NCPCs by charging fees for NCPC services. Even though
on several occasions NCPCs have been able to benefit from participation in multi-country
specific projects, this has been done at the periphery of the Programme. As further

179
explained in section 5.5, it is recommended that the funding model be changed to a
combination of country-based funding and thematic funding, to make programme funding
available for multi country projects on specific topics and for networking. It is strongly
recommended to issue the country based funding as block grants (as in the current
situation), while introducing competitive grants to eligible NCPCs/NCPC and possibly
other qualifying CP service providers to undertake programmatic activities on merit basis.
This would provide a transitional funding option for NCPCs to ease their transition from a
fully funded establishment stage to a financially independent operational stage. A
comprehensive listing of the detailed conclusions and recommendations is provided in
Table 7.7.

Table 7.7: Detailed conclusions and recommendations on funding model

Cluster 6. Funding Model
Conclusion Recommendation
The predominant model for funding of the The Programme should adopt a dual funding
Programme as a collection of country model at Programme and national levels: (1)
projects has hindered effective networking country-based block funding to support
and constrained the Programme in NCPCs in their establishment phase; and (2)
developing and delivering specialist services programme funding for (i) competitive grants
on a multi-country basis. to multiple eligible NCPCs and possibly
qualifying other CP service providers for
project based specialisation and/or
diversification; and (ii) networking initiatives.
Contributing Conclusions Supportive Recommendations
6.1 The efficiency and effectiveness of 6.1 The Programme should provide a broader
Programme management have been set of funding options to the NCPCs/NCPPs
compromised by its country-by-country to encourage their development and phase
funding and administration model. their gradual transition from fully-funded
6.2 The Programme has benefited from multi- during establishment stage to largely or
country results-based projects on specific completely financially independent on the
CP or CP-related topics that were provided longer run.
to some NCPCs, but funded and managed 6.2 The Programme could do so by splitting its
outside of the main UNIDO-UNEP CP financial commitments in block funding
Programme (e.g. GERIAP and D4S (secured and only available to
projects of UNEP). NCPCs/NCPPs during establishment stage)
6.3 The Programme provides only funding for and competitive grants (after establishment
the establishment stage of NCPCs and has stage to eligible NCPCs/NCPPs and other
not defined how to continue funding – part CP service providers, on a results and merit-
of – the activities of eligible NCPCs after basis). The competitive grant funding could
their establishment stage. The time and then be utilised to undertake specific
budget for the establishment stage varied activities, including for specialisation
hugely between countries, not related to and/or diversification of the NCPC/NCPP
the size or complexity of their and/or deliver Tier 2 and 3 services (see
manufacturing sector or pre-existing CP also Paragraph 7.2.8).
capacities. Several NCPCs were not able 6.3 The Programme management should define
to go through the establishment stage in specific packages of services it can provide
the allocated three year time, and to NCPCs in the network and should seek to
continued establishment operations for an standardise these with programmatic
additional 1-2 years without additional funding to improve their effectiveness and
funding. efficiency. Clustering in at least four service
6.4 The financial independence objective for packages (in addition to networking,
NCPCs during their establishment stage monitoring and administration) would
has distracted some NCPCs from their appear appropriate, i.e. institutional
intended public interest role as they are development support, core CP capacity
only able to remain active in information building, specialist CP technology support

180
 dissemination, advocacy, policy advice and training and policy support in CP-
and training with ongoing financial related fields. This could be in addition to
support from donors and/or their national specific projects on specialised and/or
governments. diversified CP topics.
6.5 Opportunities for standardisation of
service delivery to the NCPCs and peer
review and quality control among and by
the NCPCs have not been sufficiently
realised due to country-by-country
approach. This, in turn is linked to the fact
that donors tied funding to certain
countries according to their geographic
priorities.

7.2.7 Centre Model

The Programme’s concept for capacity building is to create national centres or

programmes, the NCPCs/NCPPs. This programme concept remains valid as the wider-
spread uptake of CP methods, technologies and policies is unlikely without permanent
advocacy at the national level, including the provision of a platform for developing and
sharing nationally-appropriate leading practices. The Programme’s focus on establishing
and supporting national centres is therefore supported by the findings of this independent
programme evaluation.
The prevailing model for a NCPC/NCPP is to establish within an existing host institution
with a mandate on business, technology and/or environment an independent centre to
deliver the standard package of CP services (see also under NCPC services in paragraph
7.2.6). The NCPC would ideally have an independent status within its host institutions,
with separate business plans, financial and contract administration and identity, to avoid
leaking of programme resources into the host institution (i.e. the NCPC then operates in
financial island mode (‘ring-fenced’) within the administration of the host institution). In
some cases outside the UNIDO-UNEP CP Programme, a different approach was followed
to create a new and legally independent organisation in cooperation with a consortium of
local institutions. The assumption is that the host institution (or the consortium of several
institutions) will provide in-kind and cash contributions to the operation of the
NCPC/NCPP and at the end of the establishment stage assume responsibility for
continuing the operation of the NCPC/NCPP. However the roles and responsibilities of
the host institution(s), national government and other public and private sector
stakeholders in continuing the operation of the NCPC/NCPP are not defined, and as a
result uncertainty remains about the institutional set up and operational model for the
NCPC. A variety of post support models therefore exists, including private company,
activity centre in public research institute or university, which all have different
capabilities to deliver both the private interest (typically CP assessment and technology
assessment and transfer services) and public interest (typically information dissemination,
training, advocacy etc.) roles of a NCPC.

The host institution arrangement has generally worked well during the funded project
stages, with no evidence to favour any particular kind of host institution. In several
countries however the host institution and/or national government took on commitments
for in-kind and/or cash support to the NCPC that could reasonably be expected to be
beyond their means, and hence did not materialise, leading to under-resourced NCPCs
and to considerable efforts to redefine activities and services. More emphasis should

181
therefore be placed on the ex-ante development of institutional scenarios, including risk
management with regard to host institution arrangements in the project preparation stage.

There is insufficient evidence that host institutions are indeed able to continue operation
of the NCPC. In some countries, the national government has taken over as provider of
institutional funding, in other countries the NCPC has been contracted to deliver services
for other donor funded projects, whilst in other countries the NCPC has turned into a
private company delivering commercial services. Whilst it might not be necessary or even
desirable to reject any of the post support phase models, it is desirable to plan and
monitor this process of institutionalisation of the NCPC/NCPP right from the start. To
this end the Programme should work on specific institutional tasks and milestones during
the support period, so that progress towards institutionalisation can be monitored during
the support phase (as per the supportive recommendations below in Table 7.8). The
institutional development should be controlled by the governing board, so the
recommendations on centre model are closely inter-related to those on governance and
ownership (as covered in paragraph 7.11)

Table 7.8: Detailed conclusions and recommendations on centre model

Cluster 7. Centre Model
Conclusion Recommendation
The capacity building model through The Programme should articulate
NCPCs/NCPCs is relevant, even though the institutional objectives and scenarios for a
Programme defines NCPCs by their service NCPC so that institutionalisation of the
categories without providing clear institutional NCPC can be monitored and provisions be
perspective(s) for the NCPC, both during and created to accommodate both the public
beyond their phase of institutional funding interest and private benefit functions of the
through the UNIDO-UNEP CP Programme. NCPC services over time.
Contributing Conclusions Supportive Recommendations
7.1 NCPCs/NCPPs have generally benefited 7.1 The Programme should pay more
from being hosted by an existing institution, attention to analysing the national
but several have experienced serious or even institutional context, performing risk
un-surmountable difficulties in obtaining the assessment and developing institutional
agreed in-kind and cash contributions from scenarios and risk management strategies
their host institutions which has before agreeing on a host institution and
compromised their success. its commitments for support to the
7.2 There is no evidence that either the model of NCPC/NCPP.
a host institution or independent operation of 7.2 The Programme should provide
the NCPC/NCPP is more effective and/or alternative post-support institutional,
sustainable. legal and operational models for
7.3 There is no evidence to favour the operation of NCPCs, and support the
establishment of a NCPC/NCPP in any governing boards of NCPCs with
particular type of institution (e.g. private developing a specific model for post
sector association, university or research support operation of the NCPC
institute), as long as staff benefits are to commensurate with national
some degree linked to centre performance. circumstances.
7.4 The Programme does not articulate 7.3 The Programme should pro-actively
alternative institutional arrangements and develop new modalities, other than
operational models that consider different NCPCs, to promote CP (for example CP
economic and institutional contexts in host champions that can access knowledge
countries and cater for ongoing delivery of and services from within the network).
the public interest functions of the
NCPC/NCPP.
7.5 Even though NCPCs/NCPPs typically
operate with a high degree of independence,

182
 they often remain legally part of their host
institutions, which has in several cases
created tensions with their host institutions
when entering into project agreements with
third parties.
7.6 The Programme has had an almost exclusive
focus on the establishment of NCPCs. In
some countries the establishment of an
NCPC was found to be not the most effective
way to promote CP.

7.2.8 NCPC Services

The nature of the UNIDO-UNEP CP Programme has been largely defined by its
standardised package of CP services. These were initially information dissemination and
awareness creation; training; CP assessments and in-plant demonstrations; and policy
advice, while transfer of ESTs was later added. These CP services originated from the CP
demonstration projects that preceded the establishment of the UNIDO-UNEP CP
Programme (as discussed in Chapter 1). Arguably the Programme was certainly at its
inception designed to create a permanent national entity with the capacity to deliver the
services needed for CP demonstration projects. This initial design turned out to be
successful in the early establishment and capacity building stage for the NCPCs/NCPPs,
as the first milestone for the NCPCs/NCPPs has been to demonstrate that CP is practical
and beneficial in the national context.

However upon having demonstrated the beneficial nature of CP, the CP services from the
NCPC/NCPC should increasingly accommodate national circumstances. It would in
general still make sense to continue the availability of the five standard types of CP
services, but it may not be necessary that the NCPC/NCPP is delivering all of them itself.
Some can possibly be delivered by other service providers. The presence of other
providers of CP and/or CP-related services could create a demand for development and
facilitation services to the NCPC/NCPP. The NCPC/NCPP should thus be encouraged to
define its own service delivery mode to accommodate national CP needs and pre-existing
and/or emerging national capabilities in CP and/or CP-related areas. As per the analysis
presented in Section 5.3.3, this would result in a balance of services between tier 1
(assessment and training), tier 2 (policy and technology development) and tier 3
(networking). The standard services are in principle applicable to both ‘core’ as well as
‘specialised’ and ‘diversified’ CP topics (see also the discussion on programme focus in
paragraph 7.2.4). The NCPC/NCPP should therefore also be encouraged and supported in
defining its own focus, on the basis of its own assessment of the national system of
policies, incentives, initiatives and experiences in CP and CP-related fields. As per the
discussion on focus of the Programme (in paragraph 5.3.2, and in paragraph 7.2.4) this
would result in a balance between core and diversified and/or specialised CP capabilities
and activities.

To support the positioning process Programme management could develop a standard

method for analysing the national ‘CP system' so as to identify key actors in CP and
related areas, assess their capacities and needs, assess the existing market and enabling
environment for CP, and then customise the NCPC/NCPC service model to assume an

183
appropriate niche role in this national system (35). This should also confirm whether
creating or maintaining an NCPC as a separate institution is warranted, or CP capacity
could be more effectively and efficiently delivered through a different institutional
mechanism.

The complete overview of detailed conclusions and recommendations with regard to

NCPC services is provided in Table 7.9.

Table 7.9: Detailed conclusions and recommendations on NCPC services

Cluster 8. NCPC Services
Conclusion Recommendation
The Programme has outlived its initial design of The Programme should support the
services which was based on a standard package NCPCs/NCPPs to undertake periodic
of NCPC services to be delivered through one assessments of the national status of CP,
single national centre, as countries that have to define and review their strategic niche
built CP capacity in different institutions require with service portfolios that are most
more tailor made NCPC services. appropriate and effective in their
respective national contexts.
Contributing Conclusions Supportive Recommendations
8.1 The Programme’s approach to deliver a 8.1 The Programme should adopt a more
standard package of CP services through each flexible approach to types of, and
NCPC/NCPP has been predominantly delivery modes for, CP services from
successful in the establishment and capacity the NCPCs/NCPPs to cater to the
building stage of all NCPCs/NCPPs. specific needs, opportunities and
8.2 To continue with the standard CP services does existing CP capabilities of the different
not reflect the very different national contexts. countries.
Important aspects that need to be taken into 8.2 The NCPCs/NCPPs should on a
account are for example: other providers of CP regular basis assess the current status
and/or related services; size of country; national of CP in their home countries as a
socio-economic and environmental priorities; basis to establish, refine and/or adjust
structure of the economy; size, diversity, their own strategic positioning and
technological capability and environmental service portfolios.
motivation of the manufacturing and other 8.3 This strategic positioning should
sectors, existing business and innovation include focus (the balance between
support networks; etc. core and diversified and/or specialised
8.3 The persistent use of standard service capabilities and activities) and service
categories and increasingly prescribing the mode (the balance between different
methods to be used has discouraged service tiers).
NCPCs/NCPPs from further development and 8.4 The Programme should provide
customisation of CP concepts and methods to analytical and methodological support
national circumstances (including for example to NCPCs/NCPPs for them to develop
the technical capabilities and environmental CP concepts, methodologies, practices,
and business motivations of the private sector). technologies and policies that are
adapted specifically to the national
circumstances (see also under
Excellence in paragraph 7.2.12).

35
The national CP system assessments could be performed by senior staff and/or directors of NCPCs in other countries, so
as to further enable collaboration and benefit from the skills available in the network.

184
7.2.9 Management and Monitoring

Management and Monitoring refers to the day-to-day operation of the UNIDO-UNEP CP

Programme and the monitoring and reporting of its performance against objectives and
outputs, outcomes and impacts. The common observation from the independent country
evaluations was that the Programme has at least historically been managed on the basis of
outputs, i.e. the number of training seminars, training days, CP assessments, etc. This is
partly a result of the poorly developed logical means-end relationships between activities,
outputs, outcomes and impacts and objectives, in particular, but not exclusively, at
Programme level (as discussed in the concluding section on strategy (paragraph 7.2.3). A
mind shift is needed among management of the Programme and the NCPCs/NCPPs that
outcomes and impacts matter, or in other words, success from the uptake of CP concepts,
know-how, practices, technologies and policies is ultimately the best contributor to
sustainability of the Programme and the individual NCPCs/NCPPs. The Programme
should therefore adopt an outcomes-based management model and establish a
comprehensive set of indicators to measure and/or estimate outcomes and possibly
impacts, from service delivery through the NCPC, as well as with regards to its own
institutional development and establishment of an enabling environment conducive to CP
in the country.

Sufficient resources should be reserved for programme management, based on a

monitoring system that allows regular performance checks on the progress towards
programme objectives and outcomes. This should also ensure that agreed project
structures and governance arrangements are adhered to and if necessary swift action taken
to remedy or adapt local deviations. Likewise agreed contributions from host institutions
and governments should also at least be tracked for early detection of operational
problems encountered, and as necessary, corrective interventions made.

It is also recommended that the Programme management adopts a matrix structure with
country and thematic responsibilities, which would be commensurate with the
recommended changes under focus (paragraph 7.2.4), funding model (paragraph 7.2.6)
and centre model (paragraph 7.2.7). Moreover, enhancements with regard to governance
and excellence (as covered in paragraphs 7.2.11 and 7.2.12) have ramifications for
management and reporting.

The listing of detailed conclusions and recommendations in Table 7.10 is therefore

limited to those only relevant for management and monitoring.

185
Table 7.10 Detailed conclusions and recommendations on Management and Monitoring
Cluster 9. Management & Monitoring
Conclusion Recommendation
Reporting on Programme The Programme should adopt a results--based
achievements is generally insufficient management model at Programme and national
to assess outcomes and impacts levels and develop a comprehensive system to
against Programme objectives which monitor performance in capacity building,
prevents adaptive management and institutional development and results and impacts
continuous improvement of the from CP service delivery. It should also monitor that
Programme’s performance. agreed project structures, governance arrangements
and contributions from host countries and
institutions are being achieved.
Contributing Conclusions Supportive Recommendations
9.1 Monitoring of outcomes and 9.1 The Programme should provide to the
impacts of service delivery by the NCPCs/NCPPs a common indicator framework for
NCPCs/NCPPs is under- determining outcomes and impacts at least for all
developed, which has hindered five core CP services, and assist them to set up the
adaptive management and necessary information systems.
continuous improvements in their 9.2 The Programme should provide specific resources
service delivery, and throughout for pilot outcome and impact monitoring schemes
the Programme. to establish best practice methods and indicators,
9.2 Effectiveness and specialisation of and demonstrate the usefulness of the information
programme management has been generated for improving NCPC service delivery.
compromised by a management 9.3 The indicator system should also cover institutional
model based on geographic development of the NCPC/NCPP and
distribution of back-stopping developments in the enabling environment for CP
responsibilities for the NCPCs. in the host country.
9.3 In several countries project 9.4 The Programme management should consider a
structure, governance arrangements matrix management structure for the NCPC
and/or host country and institution network, through a system of national contacts for
contributions deviate substantially each NCPC (both funded and no longer funded)
from those agreed by means of the combined with allocation of thematic
project agreement. responsibilities.
9.5 The Programme management should give greater
priority to ensuring that projects are implemented
as agreed, or otherwise amendments are endorsed
in a timely manner by Governing Board and the
host and donor governments.

7.2.10 Administration

Administration is used here as the umbrella term for contract management and
administration of budgets and expenditures. The experience at both national and
programme levels is that the administration is cumbersome and slow, and it is not
uncommon that NCPC/NCPP directors have to advance centre expenditures from
personal accounts as they are unable to obtain goods and services from their suppliers if
expenditure is directly paid by, or on behalf of, UNIDO with a significant delay (e.g.
venues for training, travel expenses, publication costs, etc.). Likewise the administrative
system provides severe limitations on the recruitment for external consultants at national
and international level and their market based remuneration.

The roots of the administrative problems appear to be two-fold. Firstly there is great
misunderstanding about the administrative requirements in the early stages of
establishment of the NCPC/NCPP, largely because administrative requirements have not

186
been properly clarified during project preparations (and host institutions and counterparts
are therefore not familiar with UNIDO procedures). Most NCPCs/NCPPs manage to get
through this settling in process, albeit with significant delay and frustration and with
patience from UNIDO programme management and country representatives. Secondly,
on an ongoing basis the administrative burden is high, and a serious effort should be made
by the UN agencies involved to determine whether alternative administrative
arrangements based on performance and/or against pre-determined milestones might be
possible. The UNIDO country offices and/or representatives were generally well engaged
with the NCPC/NCPP in the visited countries and played constructive roles in easing the
administrative burden. For one country however a follow up independent financial audit
has been recommended as this Programme evaluation was not tasked nor resourced to
investigate whether or not complaints were well-founded or not (Mozambique). The
comprehensive set of detailed conclusions and recommendations is provided in Table
7.11.

Table 7.2.11: Detailed conclusions and recommendations on administration

Cluster 10. Administration
Conclusion Recommendation
The UNIDO CP Unit and NCPCs/NCPPs The Programme management should
have ultimately been able to meet streamline programme administration and
administrative requirements, including shift to the extent feasible financial
financial administration and contracts’ responsibility and accountability to the
management and disbursement of funds, NCPCs/NCPPs and/or national stakeholders.
but repeatedly not in a timely manner.
Contributing Conclusions Supportive Recommendations
10.1 Most of the staff resources for CP at 10.1 The Programme management should
UNIDO were used for project develop practical ways to make programme
implementation including micro- administration less time consuming and
administration of the set-up of increase results-based accountability (e.g.
NCPCs/NCPPs and other projects. checklists, budget and expenditure
10.2 The UNIDO CP Unit faces several worksheets, quick reference guide etc).
systemic constraints, including exclusive 10.2 The Programme management should
application of agency execution, head- consider for each of the visited countries
quarter centred mode of UNIDO technical individually which steps need to be taken to
cooperation and limits on recruitment and improve administration of the NCPC (as
remuneration of consultants. per the findings in the respective country
10.3 The country visits revealed that while in reports).
most cases where UNIDO had a local
presence, it was effectively engaged with
the NCPC/NCPP and instrumental in
easing the administrative burden for the
NCPC/NCPP.
10.4 The independent country reviews found
grounds to recommend that a
comprehensive financial audit be
undertaken for Mozambique to confirm
that adequate financial control was
exercised through the UNIDO system.

187
7.2.11 Governance and Ownership

Governance should ensure accountability and transparency in the highest level decision
making on programme strategy and oversight for its implementation. Greater
accountability and transparency is in turn likely to foster ownership of activities and
results by beneficiaries, and thereby contributes to the sustainability of the NCPC as an
institution and of the CP concept and services. The current governance arrangements for
the UNIDO-UNEP CP Programme are unclear both at the Programme level as well as for
several countries at the national level. At the national level, Programme management has
promoted the establishment of boards for the NCPCs. But these had few decision making
powers and were structured as reference or steering committees for providing input and
feed back on the implementation of the UNIDO project that funded and implemented the
establishment of the NCPC. This is fundamentally different from a board accountable for
the establishment and long term success of a nationally owned centre. At the Programme
level no governance mechanism has been established for input from NCPCs and their
national public and private sector stakeholders into the programme strategy and priorities
for its implementation, even though consultations have taken place on an irregular and ad
hoc basis through for example the Directors’ meetings. It is therefore strongly
recommended that an accountable and transparent governance structure be established.
This can foster ownership of the Programme and national centres, and will reflect that
NCPCs are partners for the UN agencies and donors for the long run and that they cannot
be used as vehicles for the introduction of new services considered relevant by UN
agencies and/or donors. A comprehensive listing of the detailed conclusions and
recommendations in regard to governance and ownership is provided in Table 7.12.

At Programme level a governing board could be established comprised of elected or

appointed private and public sector representatives from host countries (for example one-
third of the membership of the board), representatives of the NCPCs/NCPPs (both
currently funded as well as no longer institutionally funded NCPCs, for example one-
third of the membership of the board) and representatives of the UN agencies and donor
governments (for example one third of the membership of the board), with an
independent chair. The Programme Management unit could then have an ex-officio, non-
voting role in this Programme Governing board. A similar board structure could be
replicated at the national level, for example with one-third membership from private
sector, one third membership from public sector, and one third membership from
academia and/or other NGOs, with an independent chair. The NCPC and possibly the
local representations of the donor governments and UNIDO could then assume ex-officio,
non voting roles in these national boards, to avoid a conflict of interest with their
administrative and executive responsibilities. The Boards should meet regularly to ensure
effective engagement, for example on a 2-3 months schedule at national level and a 4-6
months schedule at programme level.

The governing boards should consider establishing advisory boards, to seek non-binding
advice and feed back from a broader cross section of stakeholders. The advisory boards
can then also be used as a mechanism to achieve greater institutional buy-in to the
Programme, NCPC and CP concept, from the organisations that employ the members of

188
the advisory board. At national level, the advisory boards could meet regularly (e.g. 1-2
times annually), whereas the international advisory board may not have to meet in person
(or alternatively could be invited to attend Directors’ meetings and then have an advisory
board meeting piggy-backed to the Directors’ meeting).

The governing boards should be supported by efficient management structures. Several

NCPCs have made good progress in setting up internal management structures with
delegated responsibilities, whilst others still largely depend on the micro-management by
the Director. At Programme level, the management structure requires extra attention to
ensure that day-to-day roles of UNIDO and UNEP (and possibly other agencies) are
properly defined. It may therefore be instrumental to adopt a matrix management
structure with national project managers (for institutional funding to selected NCPCs
during their establishment stage) and capability leaders (for multi-country targeted
initiatives that are funded on a competitive basis from programmatic funding) (see
discussion in Section 5.5 and detailed conclusions with regard to funding model
(paragraph 7.2.6) and programme management (paragraph 7.2.9).

Table 7.12: Detailed conclusions and recommendations on governance and ownership

Cluster 11. Governance & Ownership
Conclusion Recommendation
The Programme has not established a The Programme and the NCPCs should
transparent and accountable governance adopt transparent and accountable
structure for gathering feed back from governance structures at Programme and
stakeholders, beneficiaries and NCPCs into national levels, preferably with small boards
its strategic planning and ensuring adequate with participation of private sector,
oversight over implementation of the government and civil society, that assume
Programme. The governance of NCPCs is of accountability for the success of the
varying effectiveness, accountability and Programme and the NCPCs.
transparency.
Contributing Conclusions Supportive Recommendations
11.1 Programme Management (through 11.1 Programme Management and donors
UNIDO) typically had a strong influence should adopt a participatory
on the strategies of the NCPCs in their implementation model for the NCPC
establishment stages. The influence has Programme to ensure effective
become very limited for some NCPCs, contributions from relevant public and
especially after direct financial support private sector stakeholders in planning and
through the Programme has ceased. oversight, and foster local ownership of
11.2 Most NCPCs achieve some engagement the NCPCs and the Programme.
of government and the private sector in 11.2 Programme management should define a
their planning and ongoing governance, strategy how to continue support ’mature’
but in many countries the boards or NCPCs, once they do not receive further
steering committees are top heavy, not funding through UNIDO.
accountable and in-effective in providing 11.3 NCPCs should adopt accountable and
overall guidance for the development of transparent governance structures and
the NCPC. decision making procedures, preferably
11.3 No mechanism has been established for with small boards of representatives of
NCPCs, and their national stakeholders in private, public and civil sectors, which
government and private sector, to assume accountability for the success of
influence Programme Strategy and the NCPC, and are possibly supported by
arrangements and priorities for its broader based advisory committees.
implementation. 11.4 The Programme should adopt accountable
11.4 UNIDO, UNEP and donors have and transparent governance structures and
cooperated in the programme mainly on decision making procedures, preferably
an ad-hoc basis. No coordinating headed by a board comprised of

189
 mechanism was in place and no representatives of NCPCs and public and
programme management tools have been private sectors in host countries and of
applied to ensure that the inputs of all relevant international agencies and donors,
stakeholders contribute to the programme with ex-officio membership of Programme
objectives. Management.
11.5 A joint Programme Management
mechanism should be established under
the leadership of UNIDO and UNEP, with
input from Donors and other relevant
agencies.

7.2.12 Excellence

The Programme has an inherent ambition for excellence and desire for the NCPCs/NCPPs
to become centres of excellence. The Programme is consistently being marketed as
‘holistic’ and ‘integrated’, as distinctive features to other CP or CP-related initiatives.
This evaluation however found that there are no specific mechanisms in place to drive
and deliver excellence in CP service delivery. Even standard professional practices were
not adhered to for a number of products and services from several of the visited
NCPCs/NCPPs. The diversity of international consultants and reference centres that
provide inputs to the Programme is limited, and the Programme management is not
resourced for effective quality control over services provided by consultants and/or
NCPCs/NCPPs. Overall it does appear that the Programme is at risk of becoming
complacent.

It is therefore urgently required for the Programme to establish a culture of

experimentation and continuous improvement in CP service delivery. There are different
options for doing so, including strengthening of the professional and intellectual CP
leadership in the Programme management, providing training and coaching support in
non-technical professional disciplines, benchmarking, diversification of consultant inputs,
peer review and awards. Table 7.13 provides a complete listing of the detailed
conclusions and recommendations in regard to excellence. It should also be noted that
several other clusters of recommendations could contribute to achieving excellence,
including: accountability and transparency in governance at Programme and national
levels (see paragraph 7.2.11), effective networking and opening up of network to CP
service providers not established through the Programme (see paragraph 7.2.5) and
introducing a competitive grant component in the funding model (see paragraph 7.2.6).

Table 7.13: Detailed conclusions and recommendations on excellence

Cluster 12. Excellence
Conclusion Recommendation
Despite its ambition for excellence, thematic The Programme should establish a culture of
leadership in the Programme management is experimentation and continuous
weak, as well as its incentives and improvement in CP service delivery.
opportunities for realising continuous Sufficient programme funding should be
improvements in development, adaptation made available for that purpose.
and replication of CP services and
initiatives.
Contributing Conclusions Supportive Recommendations
12.1 The Programme management has not been 12.1 The Programme management should be
sufficiently resourced to provide thematic adequately resourced to provide
and professional leadership, and for intellectual and professional CP leadership

190
Cluster 12. Excellence
effective quality review of CP service to effectively engage with directors,
delivery by NCPCs/NCPPs and boards and host institutions of
international consultants. NCPCs/NCPPs and guide these in
12.2 National stakeholders are generally achieving excellence in all aspects of their
satisfied with the quality of services service delivery.
delivered through the NCPC/NCPP. 12.2 The programme should consider offering
12.3 In the visited countries there is limited training and coaching support to further
evidence of ongoing development and professionalise NCPCs/NCPPs and ensure
customisation of CP concepts and methods best practices in communication,
to national circumstances. marketing, CP auditing, professional and
12.4 In several of the visited countries it was vocational training, advocacy and
found that the NCPC did not have adequate stakeholder engagement are being
professional capacities and systems in employed by the NCPCs/NCPPs in their
place for standardised, effective and service delivery.
efficient delivery of customised services in 12.3 The Programme management should assist
all its service areas. the NCPCs/NCPPs in benchmarking their
12.5 The effectiveness and efficiency of service approaches to communication, marketing,
delivery is compromised by insufficient professional and vocational training, CP
standardisation and absence of targeting auditing, advocacy and stakeholder
and branding of CP services in several of engagement against (international) best
the visited countries. practices (both within and outside the CP
12.6 A degree of duplication exists as NCPCs arena).
are under different projects – forced to – 12.4 The NCPCs/NCPPs should develop and
using different concepts and methods for implement a knowledge- and skills-
the same type of service. management strategy to ensure they retain
12.7 NCPCs that have established a quality (and and possibly further develop their in-house
possibly environmental) management core CP competencies.
system have benefited from this to improve 12.5 NCPCs/NCPPs should in their
their professional service delivery. establishment stage be coupled with an
12.8 Several NCPCs rely heavily on services IRC and support from their IRC should be
from external consultants for delivery of kept focused on development of core CP
their core services including CP competencies and overall coaching of
assessments. This may compromise the NCPC development. However the IRC
ability of the NCPC to advocate CP and should NOT have a dual role in also
effectively perform quality control on their administering the project and its funds.
national consultants. 12.6 A mechanism should be established for
12.9 In the visited countries the NCPCs have greater national input in selection of
generally been satisfied with the technical consultants in particular for diversified
assistance they received for developing and/or specialised service areas, setting
core CP capacities, but it should be noted their ToRs and managing their
that the amount and quality of technical performance.
assistance provided has varied hugely 12.7 The Programme management should give
between NCPCs in different countries. priority to further diversify its field of
12.10 Most visited NCPCs with an international consultants/reference centres
international reference centre appreciate as a way to encourage experimentation
the benefits of such longer lasting and excellence among the NCPCs/NCPPs.
relationship in particular in the early stages 12.8 The Programme Management (or
of NCPC establishment to support core CP preferably the Programme’s governing
capacity development, in particular CP board) should consider introducing peer
assessment and technical skills. review and reward systems to showcase
12.11 The selection of international excellence within the network of
consultants/International Reference NCPCs/NCPPs (for example an annual
Centres current active in the Programme award scheme with different categories).
have highly comparable expertise and
technical skills, and this limits exposure of

191
Cluster 12. Excellence
NCPCs/NCPPs to different ways of doing
CP as a basis for their own expertise
development.
12.12 There is a degree of dissatisfaction in
regard to specialist consultancies on CP
technologies, partially as a result of
perceived inflexibility to select consultants
and/or mismatches in expectations.

7.3 Final Remark

This independent evaluation was undertaken “to provide conclusive evidence with regard
to the current status, the potential and the needs of the NCPCs and related initiatives. It
will do this by carrying out an independent programme evaluation of the CP programme,
leading to concrete recommendations with regard to the future strategy of the
programme” (immediate objective) (36).

The current status has been described in Chapters 2 (programme review), 3 (self
evaluation) and 4 (independent evaluation), and analysed and evaluated in Chapters 5 and
6 respectively. The current status is best summarised as ‘youth’ stage. NCPCs/NCPPs
have been established and are reportedly undertaking CP and CP-related activities. There
is a richness of experience and expertise, and reasonable progress has been made in
putting CP on the agenda, delivering professional training and implementation in
particular of low to medium technology options. There are pockets of excellent work, but
also of poorer quality work, but the Programme would in principle have the potential to
effectively capture and disseminate best practices among and within the emerging
network.

The potential of the Programme is great as the relevance of CP is on the rise, due to
various factors, that each have different dynamics in the various host countries for the
Programme, which should create greater awareness and demand from public and private
sectors that the Programme can cater to. A significant performance gap [58] remains
between industry in developing countries and global best practices, so also from a
technical perspective the potential should be rated high.

The biggest challenge remains for the Programme to stand up to the challenges posed by
the changing interests and demands from governments and private sector. For this, the
Programme urgently needs a consistent Strategy that is impact-focused, delivers and
values excellence and takes due account of the specific situation of host countries. The
Strategy should drive the institutionalisation, positioning and profiling of NCPCs/NCPPs
into nationally appropriate niches with customised service and capacity profiles. It should
effectively promote the sharing of leading practices within a competence based network
of CP support institutions, including qualifying NCPCs/NCPPs and other CP service
providers not established through the UNIDO-UNEP CP Programme. The funding,
management and governance models should then also be brought in line with the
demands of a maturing Programme, including more programme- and less project-by-
project funding and a truly joint programme management by UNIDO and UNEP. NCPCs

36
ToR Independent Evaluation and Strategic Re-orientation of the UNIDO Cleaner Production Programme and related
Initiatives, UNIDO Project Document 8 March 2007.

192
will demonstrate performance against the Programme’s outcomes and impacts to continue
their association with the Programme. This vision of a strengthened and re-energised
Programme has been further expanded in the twelve sets of recommendations provided
before in this chapter.

In rounding up this evaluation the reader should also be reminded of the inherent
limitations of the evaluation methodology. Responses to the self-evaluation could not be
verified in detail, and a respondents’ bias can therefore not be excluded. The country
selection for the independent evaluations was not randomised so that results from the 18
country visits cannot be generalised as being applicable to all NCPCs/NCPPs. The
country visits were brief and even though the set of interviews with key stakeholders
enabled the evaluators to construct a picture of NCPC performance, it was not possible to
review all outputs of the respective NCPC comprehensively. Moreover, the distribution of
the country visits to the team members was also not randomised, and in combination with
the different profiles of the evaluators, there may have been an evaluator’s bias in the
independent country evaluations. Despite these limitations, the evaluation methodology
was in tune with international practices for constructive evaluations. A relative advantage
of such type of evaluations is the opportunity to gather inputs from a broad cross section
of stakeholders, including some intimately involved in the programme and some outside
participants and observers, into strengths and weaknesses of the programme and
opportunities for improvement. A drawback is that some interviewees may not have had
full information on all details of the Programme.

The information collected for this programme evaluation displayed huge diversity and
richness, and unfortunately only part of that could be brought to the fore in this main
evaluation report. It is worthwhile familiarising with the additional information that has
been compiled for the all NCPCs/NCPPs (as in the country profiles complementary to
this evaluation report) and in particular for the visited countries (in the independent
country evaluation reports that can be accessed upon request to UNIDO).

The evaluation study has achieved its output by providing an evidence basis on the status,
potential and needs of the NCPCs/NCPPs, and generating practical recommendations and
suggestions for improving the Programme. It is hoped that the planned outcome will now
also be forth-coming, namely: “UNIDO management, UNEP management, donors and
other stakeholders will use the conclusions and recommendations of the evaluation to
elaborate an evidence-based, comprehensive strategy for future assistance to and
cooperation with Cleaner Production Centres and Programmes and related initiatives
and institutions” (37). It is understood that the scope of recommendations is broad and that
evaluation and implementation of recommendations should therefore be undertaken step-
by-step.

37
ToR Independent Evaluation and Strategic Re-orientation of the UNIDO Cleaner Production Programme and related
Initiatives, UNIDO Project Document 8 March 2007 (see Annex 2).

193
Annex 1: Bibliography
1. Hirshhorn, J. and K. Oldenburg, Prosperity without Pollution: the prevention
strategy for industry and consumers. 1991, New York, USA: Van Nostrand
Reinhold. 386.
2. OTA, Serious Reduction of Hazardous Waste: for pollution prevention and
industrial efficiency. 1986, Office of Technology Assessment: Washington DC,
USA.
3. USEPA, Waste Minimisation Opportunity Assessment Manual. 1988, United
States Environmental Protection Agency: Cincinnatti (OH), USA. p. 26.
4. Sarokin, D., D. Muir, et al., Cutting Chemical Waste: what 29 organic chemical
plants are doing to reduce hazardous waste. 1985, New York, USA: Inform. 534.
5. Freeman, H., T. Harten, et al., Industrial Pollution Prevention: a critical review.
Journal of the Air and Waste Management Association, 1992. 42(5): p. 618-656.
6. USEPA, Facility Pollution Prevention Guide. 1992, United States Environmental
Protection Agency: Washington DC, USA. p. 140.
7. Dieleman, H., R. Van Berkel, et al., Choosing for Prevention is Winning, in
PREPARE Manual and Experiences, M. Crul, H. Brezet, and S. de Hoo, Editors.
1991, Ministry of Economic Affairs (for EuroEnviron): The Hague, The
Netherlands. p. 112.
8. Backman, M., D. Huisingh, et al., Preventive Environmental Protection Strategy:
first results of an experiment in Landskrona Sweden, in PREPARE: Manual and
Experiences, M. Crul, Editor. 1991, Ministry of Economic Affairs (for
EuroEnviron): The Hague, The Netherlands. p. 39.
9. UNEP, Government Strategies and Policies for Cleaner Production. 1994,
United Nations Environment Programme: Paris, France. p. 32.
10. van Berkel, R., Cleaner Production and Eco-Efficiency, in The International
Handbook of Environmental Technology Management, D. Marinova, D.
Annandale, and J. Phillimore, Editors. 2007, Edward Elgar Publications:
Cheltenham, UK. p. 67-93.
11. WBCSD, Measuring Eco-Efficiency: a guide to reporting company performance.
2000, World Business Council for Sustainable Development: Geneva,
Switzerland. p. 36.
12. WBCSD, Eco-Efficiency: creating more value with less impact. 2000, World
Business Council for Sustainable Development: Geneva, Switzerland. p. 32.
13. Johannson, L., ed. Greening on the Go: a pocket guide to green productivity.
2005, Asian Productivity Organisation: Tokyo, Japan. 278.
14. Envirowise, Waste Minimisation for Managers, in Good Practice Guides
(GG367). 2002, Envirowise: Oxford, UK. p. 24.
15. UN, ed. Agenda 21. 1992, United Nations: New York, USA.
16. WSSD, The Johannesburg Declaration on Sustainable Development and Plan of
Implementation. 2002, United Nations World Summit on Sustainable
Development: Johannesburg, South Africa.
17. UNEP, Sustainable Consumption & Production: how development agencies make
a difference. 2006, United Nations Environment Programme - Division of
Technology, Industry and Economics: Paris, France.
18. UNEP and UNIDO, Audit and Reduction Manual for Industrial Emissions and
Wastes,. 1992, United Nations Environment Programme and United Nations
Industrial Development Organisation: Paris (France)/Vienna (Austria).

195
19. van Berkel, R., J. Kryger, et al., Preliminary Experiences with the Introduction of
Cleaner Production in China and India. UNEP Industry and Environment
Review, 1994. 17(4): p. 46-50.
20. van Berkel, R., S. Chandak, et al., From Waste to Profits: The Indian Experience
(towards financial and environmental dividends from waste minimisation in small
scale industries in India). 1995, United Nations Industrial Development
Organisation: Vienna, Austria. p. 100.
21. Luken, R., R. Stevenson, et al., eds. Special Issue on Institutional Capacity for
Cleaner Production in Developing and Transition Economies. Journal of Cleaner
Production Vol. 12 (no 3). 2003.
22. Kisch, T., E. Ryden, et al., Evaluation of the UNIDO UNEP National Cleaner
Production Centres Programme. 1996, International Institute for Industrial
Environmental Economics: Lund, Sweden. p. 60.
23. Luken, R. and J. Navratil, A Programmatic Review of the UNIDO-UNEP
National Cleaner Production Centres. Journal of Cleaner Production, 2003. 12:
p. 195-205.
24. UNEP and UNIDO, Changing Production Patters: learning from the experiences
of the National Cleaner Production Centres. 2002, United Nations Environment
Programme and United Nations Industrial Development Organisation: Paris
(France)/Vienna (Austria).
25. Grütter, J., Impact Assessment of Sustainable Enterprise Development Centres.
2005, Swiss State Secretariat for Economic Affairs: Bern, Switzerland. p. 36.
26. PASS, Mid Term Review of National Cleaner Production Centre Project: final
report. 2006, PASS Research and Consultancy for Royal Norwegian Embassy
and United Nations Industrial Development Organisation: Colombo, Sri Lanka. p.
62.
27. UNIDO, Progress report: UNIDO-UNEP NCPC Programme (Project
US/Int/92/044). 1997, United Nations Industrial Development Organisation:
Vienna, Austria.
28. UNIDO/UNEP, Letter of Agreement: UNIDO-UNEP National Cleaner
Production Centres Programme. 1994, United Nations Industrial Development
Organisation/United Nations Environmental Programme: Vienna (Austria)/Paris
(France).
29. UNIDO, Project Document: UNIDO-UNEP NCPC Programme (Project
US/Int/92/044). 1994, United Nations Industrial Development Organisation:
Vienna, Austria.
30. UNIDO, National Cleaner Production Centres Programme. 2007, United
Nations Industrial Development Organisation: Vienna (Austria).
31. UNIDO, UNIDO's Holistic and Sectoral Cleaner Production Strategy:
Productivity Increase and Competitive Market Access with Cleaner Production
and Environmentally Sound Technology Transfer 2003-2006. 2003, United
Nations Industrial Development Organisation: Vienna (Austria). p. 21.
32. UNIDO, UNIDO PTC/PEM Cleaner Production Programme: Business Plan
2003-2005. 2003, United Nations Industrial Develpment Organisation: Vienna
(Austria). p. 43.
33. Skinner, J., Establishing international networks - UNEP IE/PAC experience.
1993, United Nations Environment Programme: Paris, France.
34. UNEP, Capacity Building in Cleaner Production Centres: a training resource
package. 1998, United Nations Environment Programme/Division of Industry,
Technology and Economics: Paris, France.

196
35. Baark, E. and J. Strahl, The Response of International Organisations to the
Environmental Challenge: The Case of UNIDO. Development and Change, 1995.
26.
36. UNEP, Profiting from Cleaner Production: Resource Kit for Training and
Checklists for Action. 2003, United Nations Environment Programme: Paris,
France.
37. UNEP, NCPC Survey: summary results. 2005, United Nations Environment
Programme: Paris (France).
38. UNEP, Design for Sustainability: A Practical Approach for Developing
Economies. 2006: Paris, France.
39. UNEP, Energising Cleaner Production: a guide for trainers. 2007, United
Nations Environment Programme: Paris, France. p. 74.
40. UNIDO, UNIDO Cleaner Production Toolkit. 2005, United Nations Industrial
Development Organisation: Vienna, Austria.
41. UNIDO, Training Kit on Cleaner Production Policy. 2002, United Nations
Industrial Development Organisation: Vienna, Austria. p. CD Rom.
42. UNIDO, Chemical Leasing: business models. 2006, United Nations Industrial
Development Organisation: Vienna, Austria.
43. UNEP, Energy Efficiency Guide for Industry in Asia 2006, United Nations
Environment Programme: Bangkok, Thailand.
44. UNEP, Sustainable Consumption and Production (SCP):Making the Connection.
2006, United Nations Environment Programme: Paris, France.
45. UNEP, Environmental Agreements and Cleaner Production: questions and
answers. 2006, United Nations Environment Programme: Paris, France. p. 28.
46. UNEP, Environmental Technology Assessment. 2000, United Nations
Environmental Programme. p. 100.
47. UNEP, Advancing Sustainable Consumption in Asia: a guidance manual 2005,
United Nations Environment Programme: Paris, France. p. 74.
48. UNEP. UNEP/IAPSO Product Criteria Database for Sustainable Public
Procurement. 2006 [cited; Available from:
http://www.uneptie.org/pc/sustain/policies/green_find.asp.
49. Brezet, H. and C. van Hemel, Ecodesign: a promising approach to sustainable
production and consumption. 1997, Paris, France: United Nations Environment
Programme.
50. UNEP/Wuppertal Institute, Assistant: measure and improve your business
performance month by month, in The Efficient Entrepreneur Calendar. 2000,
United Nations Environment Programme/Wuppertal Institute Paris,
France/Wuppertal, Germany. p. 80.
51. UNEP/CSCP, Policy Instruments for Resource Efficiency: towards sustainable
consumption and production 2007, United Nations Environment Programme:
Wuppertal, Germany. p. 124.
52. Grütter, J., M. Kellermann, et al., Evaluation Report NCPC South Africa. 2004,
United Nations Industrial Development Organisation: Pretoria/Vienna (South
Africa/Austria). p. 60.
53. Klarer, J., T.M. Chi, et al., Report of the Independent Joint In-Depth Evaluation
Mission: Vietnam National Cleaner Production Centre. 2003, United Nations
Industrial Development Organisation: Vienna, Austria. p. 76.
54. CNCPC, Introduction to Enterprise Cleaner Production Audit Manual. 1995,
China National Cleaner Production Centre: Beijing, P.R. of China.
55. OECD, Managing National Innovation Systems. 1999, Organisation for
Economic Co-operation and Development: Paris, France.

197
56. OECD, Governance of Innovation Systems: Volume 1 Synthesis Report. 2005,
Organisation for Economic Co-operation and Development: Paris, France. p. 120.
57. Foxon, T. and R. Kemp, Innovation Impacts of Environmental Policy, in The
International Handbook of Environmental Technology Management, D.
Marinova, D. Annandale, and J. Phillimore, Editors. 2007, Edward Elgar
Publications: Cheltenham, UK. p. 119-139.
58. Luken, R. and F. van Rompaey, Environment and Industry in Developing
Countries: assessing the adoption of environmentally sound technology. 2007,
Cheltenham, UK: Edward Elgar Publishing. 345.
59. USEPA, An Organisational Guide to Pollution Prevention. 2001, United States
Environmental Protection Agency: Cincinnati (OH), USA.
60. Chandak, S., R. Wadhwa, et al., From Waste to Profits: guidelines for waste
minimisation. 1994, National Productivity Council: New Delhi, India. p. 50.
61. de Hoo, S., H. Brezet, et al., Manual for the Prevention of Waste and Emissions,
in PREPARE Manual and Experiences, M. Crul, Editor. 1991, Ministry of
Economic Affairs (for EuroEnviron): The Hague, The Netherlands. p. 84.
62. UNEP/UNIDO, Audit and Reduction Manual for Industrial Emissions and
Wastes, in Technical Report Series No 7. 1992, UNEP Industry & Environment
Office/UNIDO: Paris, France.
63. van Berkel, R. and M. Lafleur, Application of an Industrial Ecology Toolbox for
the Introduction of Industrial Ecology in Enterprises. Journal of Cleaner
Production, 1997. 5(1-2): p. 27-38.
64. van Berkel, R., E. Willems, et al., Development of an Industrial Ecology Toolbox
for the Introduction of Industrial Ecology in Enterprises. Journal of Cleaner
Production, 1997. 5(1-2): p. 11-26.
65. Stone, L., Limitations of cleaner production programmes as organisational
change agents. II. Leadership, support, communication, involvement and
programme design. Journal of Cleaner Production, 2005. 14(1): p. 15-30.
66. Stone, L., Limitations of cleaner production programmes as organisational
change agents I. Achieving commitment and on-going improvement. Journal of
Cleaner Production, 2005. 14(1): p. 1-14.
67. Johannson, L., ed. Handbook on Green Productivity. 2005, Asian Productivity
Organisation: Tokyo, Japan. 340.
68. WorldBank, Greening Industry: new roles for communities, markets and
governments. 1999, New York, USA: Oxford University Press.
69. Mertz, B., O. Davidson, et al., eds. Methodological and Technological Issues in
Technology Transfer. 2000, Cambridge University Press for Intergovernmental
Panel on Climate Change: Cambridge, UK.
70. Trindade, S., T. Siddiqi, et al., Managing Technological Change in Support of the
Climate Change Convention: a framework for decision making, in
Methodological and Technological Issues in Technology Transfer, B. Mertz, et
al., Editors. 2000, Cambridge University Press for Intergovernmental Panel on
Climate Change: Cambridge, UK. p. 49-66.
71. Halls, S., Technology Transfer and Uptake of Environmentally Sound
Technologies, in The International Handbook of Environmental Technology
Management, D. Marinova, D. Annandale, and J. Phillimore, Editors. 2007,
Edward Elgar Publications: Cheltenham, UK. p. 174-191.
72. van Berkel, R., Scoping Paper on Methodology and Policy Issues in Development
and Adoption of Environmentally Sound Technologies. 2007, United Nations
Economic and Social Commission for Asia and Pacific: Bangkok, Thailand. p.
38.

198
Annex 2: Terms of Reference

Terms of Reference

Independent Evaluation and Strategic Re-orientation of

the UNIDO Cleaner Production Programme and related
Initiatives
8 March 2007

Starting date: March 2007

Duration: 7 months
UNIDO HQ, 40 countries with Cleaner Production Centres, 17
Project site: field visits

Executing agency/ UNIDO (executing)/ UNEP (cooperating)

cooperating agency: Implementation by OSL/EVA and ECB/CPU

Brief description:
In the proposal made by the Director General to the Industrial Development Board in its 32nd
session with regard to the UNIDO Medium Term Programme Framework, it is envisaged to
“take cleaner production and energy efficiency activities to a new level. With respect to its
National Cleaner Production Centres (NCPCs), UNIDO will strengthen the existing network,
introducing quality and performance criteria and opening it to other, bilaterally funded, cleaner
production centers that meet its criteria. It will strongly promote it as a global delivery platform
of excellence for the implementation of sustainable industrial development activities.” To set a
basis for the new strategy, the technical departments of UNIDO (PTC/ECB) and UNEP in
cooperation with the major donors (Switzerland, Austria) of the Cleaner Production Programme
have decided to carry out an independent thematic evaluation of the ongoing Cleaner Production
centres and related initiatives.

The present document provides the terms of reference for this evaluation and includes the
immediate steps to follow up on evaluation findings and recommendations. The former part will
be implemented by OSL/EVA, the latter by ECB/CPU.

199
A. CONTEXT

The UNIDO Corporate Strategy (38) considers the existing international network of National
Cleaner Production Centres (NCPCs) and Programmes (NCPPs) an effective vehicle for the
promotion and implementation of UNIDO’s programmes, especially in the field of cleaner
production and related issues, benefiting from the presence of reliable and trained focal points in
the countries.

Since its inception in 1994, 35 NCPCs and NCPPs have been established within the
UNIDO/UNEP Cleaner Production Programme. More recently, one Regional Cleaner Production
Programme for Latin America and the Caribbean, with 14 countries participating was created. The
“bilateral” Cleaner Production Centres established by bilateral donors (in particular the Swiss State
Secretariat for Economic Affairs (SECO), and the German GTZ) have maintained close relations
to the UNIDO/UNEP CP Programme. The CPCs, depending on the level and duration of UNIDO
support, the support they receive from national and international institutions, the demand for
cleaner production in their countries and the success they have had in positioning themselves as a
leading agency for environmental matters related to industry, show different levels of institutional
capacity and have different needs for future support.

CPCs, after the initial period of UNIDO (or bilateral) assistance, develop into national (private
and/or public) institutions with their own local ownership structure. As a result, the level of
information in UNIDO with regard to the needs for future assistance, potential for cooperation
with other institutions, financial and institutional sustainability, strengths and weaknesses in the
different service areas (plant level assessments, policy, training, etc.) varies and is in many cases
limited.

The planned evaluation will assess the needs, capacities and potential of NCPCs in order to
provide feedback regarding the effectiveness, efficiency, relevance and sustainability of the
assistance provided so far. This will provide the stakeholders of the programme with a sound basis
for the design of the future cooperation strategy as envisaged in the Medium Term Planning
Framework 2009-2011.

B. REASONS FOR UNIDO ASSISTANCE

UNIDO is the lead agency of the UNIDO/UNEP Cleaner Production Programme and responsible
for the setting up of and support to the National Cleaner Production Centres. Existing working
relations between UNIDO and stakeholders involved in NCPCs will facilitate access to
information. The overall coordination role of UNIDO within the international CP activities makes
UNIDO the ideal coordinator of this evaluation.

C. THE PROJECT

C.1. Objective of the project

A more effective Cleaner Production Programme of UNIDO and partner agencies, based on a
strengthened network of cleaner production centres and programmes.

38
“Operationalizing UNIDO’s Corporate Strategy – Services and priorities for the medium term, 2004-2007”

200
To achieve this objective, the project will aim at providing conclusive evidence with regard to the
current status, the potential and the needs of the NCPC and related initiatives. It will do this by
carrying out an independent programme evaluation of the CP programme, leading to concrete
recommendations with regard to the future strategy of the programme.

Particular emphasis of the evaluation will be given to a number of criteria39 to assess the capacities
and the potential of individual centres to form part of a strengthened and effective global network
of CP institutions.

The closure component of the evaluation will be the UNIDO / UNEP Cleaner Production Annual
Meeting. The meeting will bring together representatives from the National Cleaner Productions
Centres and Programmes, technical institutions and consultants, international organizations, donors
and other stakeholders involved in the Cleaner Production projects and programmes.

During the Annual Meeting, the results of the evaluation of the UNIDO / UNEP CP Programme
will be discussed and the lessons learned from this experience will be further analyzed. Based on
the outcome of the discussion and the innovative ideas presented during the Annual Meeting, the
work plan and strategy of the UNIDO / UNEP CP Programme for the upcoming years will be
finalized and approved.

C.2. The UNIDO approach

Institutional arrangements:

UNIDO Evaluation Group (OSL/EVA) will be responsible for overall project management and
backstopping and for the implementation of outputs 1 to 5, which form the independent evaluation.
The evaluation team will work under the supervision of OSL/EVA and consist of three
international experts in the field of cleaner production including a Team Leader. National experts
in the countries to be covered by a field visit will support the evaluation team in their work. The
independent evaluation will be carried out in accordance with UNIDO evaluation policy.

UNIDO Cleaner Production Unit (PTC/ECB/CPU) will be responsible for the follow up on
findings and recommendations, i.e. output 6. For that purpose the annual NCPC meeting 2007
forms part of the overall project, since the discussion of future strategy will take place in the
course of this meeting.

The evaluation team will be guided by a steering committee composed of one representative from
each of the institutions participating in the evaluation: UNIDO OSL/EVA (chair) and PTC/ECB,
UNEP, Switzerland, Austria, GTZ.

It will meet three times over the project period:

1. to decide on the assessment criteria applied in the evaluation,
2. to select the countries for in-depth assessment based on the portfolio analysis,
3. to discuss findings and preliminary conclusions, recommendations and lessons learned
based on the draft report.

39
see Annex III for a list of example criteria

201
One of the international consultants will act as Team Leader coordinating the report writing with
the other two international consultants. The content of the evaluation report will come under full
responsibility of the evaluation team, with evaluators acting in their personal capacity as
evaluation experts. Comments, suggestions and recommendations from project stakeholders,
including the members of the steering committee, will be taken into due consideration by the
evaluation team.

Coverage:

x All NCPCs and NCPPs under the UNIDO/UNEP programme (35).

x Regional cooperation initiatives among NCPCs (1, Latin America).
x All bilateral CPCs supported by SECO (Colombia, Peru, Bolivia, Jordan).
x Other donors’ CP Centres/programmes (e.g. GTZ), to be decided by the steering
committee based on relevance of such centres/programmes for the UNIDO/UNEP
network.
Evaluation Methodology:

To carry out a forward looking strategic assessment of performance, capacities and future potential
of CPCs, the evaluation exercise will encompass the following steps:

1. Document review: elaborate a set of criteria for the assessment (such as: financial
sustainability, institutional sustainability, human resource capacity, client structure, service
capacity, etc.) of CPCs. This will be based on a thorough review of existing documentation
on activities, performance and capacities of CPCs.

2. Obtain information on the established criteria for all CPCs covered by the evaluation. This
will be done through a self-evaluation exercise to be carried out by each CPC together with its
counterpart (host) organization(s) and main stakeholders. Information gaps will be closed
through telephone interviews.

3. Carry out a portfolio analysis of existing CPCs with regard to:

x Needs for future assistance

x Potential for cooperation with other institutions

x Financial and institutional sustainability

x Strengths, Weaknesses, Opportunities and Threats (SWOT) in the different service areas
(plant level assessments, policy, training, etc.)

x Other criteria to be established during the evaluation process.

At the end of this step, different types of CPCs will be described.

4. Select a representative sample of CPCs for in-depth performance assessment through field
missions. A number of working hypotheses will be elaborated by the evaluation team and the
steering committee. These hypotheses will resemble future strategy options for UNIDO,
UNEP, donors and other stakeholders and will be tested through the field missions to selected

202
 CPCs. The requirements of stakeholders for the evaluation of particular NCPCs will be taken
into consideration. NCPCs in Central America and South Africa will be included in the field
visit programme given the overdue evaluation of these NCPCs.

5. Assess the performance and capacities of selected Cleaner Production Centres. Selected CPCs
should be representative for the different types of CPCs established under step 3.

6. Synthesis of results from step 1 (document review), 2 (self evaluation), 3 (portfolio analysis)
and 5 (in-depth performance assessment) into an evaluation report including conclusions,
recommendations and lessons learned. This step will include an analysis, at the programme
level, of the relevance, the effectiveness and the efficiency of the NCPC programme.

7. Management response: collect responses to the recommendations, including envisaged steps

towards their implementation, from the management of the main stakeholders of the
evaluation (UNIDO, UNEP, donors).

8. Presentation and discussion of the evaluation results at the Annual NCPC Meeting in
September 2007

C.3. RBM code and thematic area code

RBM code: B.2.3

Thematic Area Code: EAE

C.4. Expected outcomes

UNIDO management, UNEP management, donors and other stakeholders will use the conclusions
and recommendations of the evaluation to elaborate an evidence-based, comprehensive strategy
for future assistance to and cooperation with Cleaner Production Centres and Programmes and
related initiatives and institutions.

The new strategy will provide the basis for a strengthened global network for the promotion of
cleaner production.

C.5. Outputs and activities

Outputs and activities under OSL/EVA responsibility:

Output 1: Assessment Criteria
Activities Responsibility
1.1 Collect coherent set of information for each of the centres and UNIDO CP
programmes covered by the evaluation Unit
1.2 Review of documentation on centres and programmes Evaluation
Team
1.3 Set of assessment criteria established Evaluation
Team
1.4 Meeting of Steering Committee to approve criteria Steering
Committee

Output 2: Self Evaluation

203
 Activities Responsibility
2.1 Design format for self evaluation based on assessment criteria Evaluation
Team
2.2 Send self evaluation format to all Centres and Programmes covered by the Evaluation
evaluation Team
2.3 Provide assistance and follow up to Centres and programmes in Evaluation
conducting the self evaluation Team

Output 3: Portfolio Analysis

Activities Responsibility
3.1 Analyse information collected under output 1 and output 2 and write a Evaluation
first input report as a basis for portfolio analysis Team
3.2 Describe the existing portfolio of CP centres and programmes by Evaluation
identifying different types or categories of centres/programmes Team
3.3 based on the portfolio analysis, select centres/programmes for in-depth Evaluation
performance assessment Team /
Steering
Committee

Output 4: In-depth performance assessment

Activities Responsibility
4.1 Field visits including interviews of beneficiaries and stakeholders Evaluation
Team
4.2 Write brief evaluation reports for each centre/programme visited Evaluation
Team
4.3 Write summary report for the in-depth assessment Evaluation
Team

Output 5: Conclusions, Recommendations and Lessons Learned

Activities Responsibility
5.1 Based on in-depth assessments, document review and portfolio analysis Evaluation
write evaluation report and draw conclusions, recommendations and lessons Team /
learned Steering
Committee
5.2 Management response to recommendations OSL/EVA

Outputs and activities under ECB/CPU responsibility:

Output 6: New strategy for the Cleaner Production Programme based on a strengthened
network of NCPCs and related initiatives
Activities Responsibility
6.1 Meeting of all stakeholders to discuss evaluation conclusions, ECB/CPU
recommendations and lessons learned (Annual NCPC meeting)
6.2 Draft strategy paper / circulation / feedback ECB/CPU
UNEP
6.3 Final strategy paper ECB/CPU
Unit/ UNEP

204
C.6. Timeline of the activities

Before the output-related activities shown below can start, experts have to be identified and
recruited, the members of the steering committee have to confirm their participation and the funds
need to be transferred to UNIDO for execution. It is estimated that these activities require at least a
one-month lead-time.

Time schedule for output-related activities:

Months
Output Activity
1 2 3 4 5 6 7
1.1

Assessment 1.2
Criteria
1.3
1.4
Steering
Committee
2.1
Self Evaluation 2.2
2.3

3.1
Portfolio Analysis 3.2
3.3
Steering
Committee
4.1
In-depth
assessment 4.2
4.3
Steering
Committee
Conclusions, 5.1
Recommendations, 5.2
Lessons Learned

6.1, (Annual
New CP strategy 6.2, Meeting
6.3 NCPCs)

C.7. Risks

The principal risk of the project is that a lack of relevant information could limit the credibility and
usefulness of the evaluation’s conclusions for the envisaged strategy building. However, previous
evaluations have shown that many NCPCs do have a relatively good information base.

D. MONITORING, REPORTING AND EVALUATION

Monitoring of progress in implementing the project will be carried out by OSL/EVA on a

continuous basis. The steering committee of the project will receive status reports prior to each of
the three meetings planned over the implementation period of the project. The reports will provide
information on progress towards the objective and the expected outcomes of the project. They will
also summarize the activities carried out. No evaluation is foreseen.

205
206

Annex 1: Logical framework

Intervention logic Objectively verifiable indicators Sources of Assumptions

verification
Development A more effective Cleaner Production Programme of UNIDO and partner x Increased visibility of NCPCs and NCPPs Thematic
goal/impact agencies, based on a strengthened network of cleaner production centres and x Increased use of centres for implementation evaluation to be
programmes. of multilateral and bilateral programmes in the carried out in 2011
area of sustainable development
Outcome(s)/im UNIDO management, UNEP management, donors and other stakeholders will Incorporation of evaluation’s recommendations CP strategy of CP will remain an
mediate use the conclusions and recommendations of the evaluation to elaborate an and lessons learned in future CP strategy UNIDO, UNEP, important area of
objective(s)/ evidence-based, comprehensive strategy for future assistance to and SECO, other cooperation for
cooperation with Cleaner Production Centres and Programmes and related partners UNIDO and other
initiatives and institutions. partners involved
Outputs 1. Set of criteria for the assessment of Cleaner Production Centres and 1. Relevant set of criteria available
programmes established 2. Self assessment of performance, needs and
2. CPCs and CP programmes have carried out a self evaluation process and potential available for all CPCs covered by
are aware of their needs, potentials, strengths & weaknesses, expectations the evaluation
from cooperation 3. Different types of centres/programmes
3. Portfolio analysis of CPCs and CP programmes identified
4. In-depth performance assessment of selected CPCs and programmes 4. Coherent set of assessment reports available
5. Conclusions, recommendations an lessons learned for all visited centres and programmes
6.New strategy for the Cleaner Production Programme based on a strengthened 5. Relevant conclusions and recommendations,
network of NCPCs and related initiatives based on evidence found during evaluation,
available, Set of lessons of wider applicability
for UNIDO and stakeholders available
6. Draft strategy paper which incorporates
recommendations and lessons learned from
the evaluation
Main x Document review
Activities x Facilitate self evaluation processes of centres and programmes covered by evaluation
x Prepare a first input report as a basis for portfolio review
x Country visits to selected centres and programmes including interviews of beneficiaries and stakeholders
x Prepare draft evaluation report and collect feedback from stakeholders (management response)
x Prepare final evaluation report
x Meeting of all stakeholders to discuss evaluation conclusions, recommendations and lessons learned (Annual NCPC meeting)
x Draft strategy paper / circulation / feedback

207
Printed in Austria
V.08-58169—November 2008—100

UNITED NATIONS INDUSTRIAL DEVELOPMENT ORGANIZATION

Vienna International Centre, P.O. Box 300, 1400 Vienna, Austria
Telephone: (+43-1) 26026-0, Fax: (+43-1) 2692669
E-mail: [email protected], Internet: http://www.unido.org