Problem Definition and Theory of

Change: What For?

WHAT IS A POLICY?
As EC Officers, you are involved in a process of policy making. A public policy
can be defined by:
‣ its content (a government programme of actions in a given sector);
‣ by the choices made by governments (policies are “what government chooses to
do or not to do” DYE, 1972);
‣ by its logic (a set of policy means contributing to policy goals LASSWELL, 1958);
‣ and also by its process.
The “policy cycle” is a popular way to represent the different stages of a policy
design process. It was first used by Harold Lasswell, but many others have
been using this image since then HOWLETT 2010.
In the policy cycle, there is a problem at stake that
comes to the attention of a government, and that it
wishes to solve; the government drafts options to do so,
and decides whether it will act or not, and how. It then
implements the policies, by a series of decisions,
acts, funding, etc. It monitors the
policy unfolding and, eventually,
appraises the results (was the prob-
lem solved?) through evaluation.

💡 The policy cycle is a theoretical in-

strument to think about policy design. Take
it with a grain of salt: first, it makes the
(somewhat contestable) assumption that policy making
is always a rational endeavour to solve problems. Second, it
considers that this process is linear and sequential, which is far
from being true either. Éric Monnier talks of a “policy whirlpool” to describe
the iterative process of policy design MONNIER ET AL. 1992. Third, it considers poli-
cies as something governments do, though we know that policies in practice
are as much what organisations and people decide to do in reaction to govern-
ment decisions than what governments do PAWSON AND TILLEY 2004. But it is a useful
model to think in terms of policy design (what is needed to actually design
policies?) and evaluation.
WHAT IS THE PROBLEM?
If policy is a problem-solving activity, being able to explain what is the prob-
lem is the key to the whole policy design process.
A public problem can be defined as “any failing situation which affects a society
or a part of it by an imbalance, a tension, the inadequacies or frustrations that
it created and which provokes a collective action to rectify, mitigate or im-
prove it” REZSÖHAZY, 1996.
Not all problems lead to policy action, and the fact that a problem be-
comes a subject of political debate is not always related to its importance. But
there are “windows of opportunity” KINGDON, 1995, moments when different
“streams” collide, i.e. when there is a shared consciousness that there is a prob-
lem at stake; stakeholders advocating for solutions; and a political opportunity
to deal with this issue.
An important aspect question of policy design is then how to make the best of
such moments (which are short-lived and far apart).
The complicated part is that problems are “framed” in a certain way
when the policy design process starts, which means that there is a certain view
that is imposed on what the problem is, what causes it and what are its conse-
quences.
This “framing effect” is possible, first because real-life problems (think “crime”,
“poverty” or “global warming”) are extremely complex and entangled; their
causes are multiple, often not linear; they cannot be totally embraced and
therefore it is extremely hard to understand them, let alone fix them. They are
said to be “wicked” RITTEL & WEBBER, 1973.
So, one has to find an angle of attack. The thing is, governments tend to
think in terms of the instruments they have at their disposal and their area of
jurisdiction BARDACH, 1980. This means that problems are also framed by the solu- At the EC level, whether a
problem has to be dealt
tions that are at the government disposal (“If you have a hammer, everything with by DG Environment or DG
looks like a nail”). Stakeholders can try to influence problem definition by Agri is likely to lead to very
different solutions.
proposing solutions.
The fact also is that public problems are not objective phenomena: what is a
problem for some or at a given moment is not for others (think: sex equality).
They are social constructions, which means they are shaped by people and or-
ganisations, whose views and actions are, in turn, shaped by problem definition.
This also entails that problem definition is subject to political debates, as well
as pressures and interests of the economically and politically powerful, etc. (on
this topic and the implications for evaluation, see ARCHIBALD, 2019).
The consequence is that the problem definition process is not only a construc-
tion process, but also a deconstruction process, in which it is necessary to under-
stand how a problem is represented, as well as the consequences of this repre-
sentation.
For instance, initially framing the problem of “climate change” as an issue of For a critique of climate
individual behaviours, market equilibrium, or technological solutions will have change policies, see
SOMERVILLE, 2019
different implications than a focus on the fossil-fuel industry, global imbalance
and equity.
There are different ways to do so, e.g. through a participatory (by asking the
different stakeholders, including the silent, less powerful ones, how they view
the problem or are affected by it) or a critical approach (for an example, see
Bacchi’s “What’s the problem represented to be” approach 2009).
In the end, the deconstruction of the problem through its representation, but
also through its components, especially its different root causes will help pro-
viding alternatives in the angles of attack that governments can use to address
it: e.g., rather than trying to address “crime”, or to address it through precon-
ceived solutions, it will be possible through the problem definition process to
identify a (limited) series of crime drivers on which a government can actually
act, with different expectable consequences.

💡 Problem definition is also a key aspect of evaluation. First, analysing how

the problem has been framed is a good way to uncover the values underlying
a policy, which can then be used for critical appraisal and judgement; second, For instance, a policy
aiming first at increasing
clarifying how a problem was initially framed and how it is framed at the agricultural production may
have seemed relevant in the
moment of the evaluation (or whether it has changed in some ways) is key in 1980s and be judged as irrele-
being able to assess the relevance and explain why stakeholders are not satis- vant 30 years later, when what
is at stake is not feeding the
fied with a policy anymore; and third, initial inadequacy in defining the prob- world anymore, but the protec-
tion of the environment.
lem is a frequent cause of ineffectiveness.

REDUCING UNCERTAINTY
When governments and other stakeholders develop solutions at the policy de-
sign stage, they have no insurance that these can effectively address the prob-
lem in question. There are several uncertainties at stake:
‣ about the nature of the problem (or the adequacy of its framing);
‣ about the intrinsic capacity of a policy to solve or mitigate a problem as well as its
implementation;
‣ about the reception of that policy by the stakeholders — how they will react to it
and whether it will contribute to achieving policy goals or not BARDACH, 1977, WILDAVSKI
1979; LIPSKY 1981;

‣ about the future: whether conditions or values will change and how they will af-
fect a policy’s outcomes… or how they are judged.
All these reasons, that describe the complex nature of policy design and im-
plementation, explain why the actual outcomes of a policy cannot be
forecast... and also why unforeseeable consequences, desirable or not, are very
likely MORELL, 2005.
In that sense, launching a new policy is akin to a “leap of faith”. This
is reinforced by administrative processes in which policy design precedes im-
plementation, without any chance for experimentation or feedback from the
field.
One could argue that policy design (as well as evaluation) aims at reducing this
uncertainty, “through the accurate anticipation of the consequences of gov-
ernment actions” HOWLETT 2010. One of the ways through which policy design can
do so is by revealing the theories about how and why a policy is
supposed to work WEISS 1995.
What is a theory (of change) here? It is the set of assumptions that are made to
explain how and why a policy would solve or mitigate a problem, through a
given sequence of events. For instance, considering how people who struggle
with literacy are more at risk of losing their job, a policy could consist in de-
tecting these people and convincing them to go through literacy classes, so
that they accept to do so, attend the classes, and improve their literacy skills.
Such a theory would include many additional hypotheses, either related to the
activities (including the quality of the class) or to the issues at stake (e.g. that
job loss is related to illiteracy and not to deeper factors such as poverty,
racism, etc. that are connected with illiteracy as well).
Policies often include objectives, which are statements about the desired conse-
quences of an intervention. But objectives belong to the field of management
(what do we want to achieve? And later: did we achieve what we wanted to?):
they are also focused on what governments can do (e.g. implementing training
sessions), what is in their control, rather than what is important to them, but
that they can only influence (increasing people skills, or that they find a job).
They cannot replace assumptions about how things will work.
In the end, the theories behind policies are too often implicit and For a history of the con-
cept of a “black box”, see
constitute a “black box”, full of “assumptions, conjectures, and other mir- ASTBURY AND LEEUW 2010
acles” ARCHIBALD ET AL. 2016. Opening the box makes it possible to exert critical
thinking on its different steps. Connell & Kubisch have identified several bene-
fits for this, including “[sharpening] the planning and implementation” of a pol-
icy; preparing monitoring; and also facilitating evaluation 1998. The simple
process of discussing the theory of change is useful in the regard, as much as
the theory itself FUNNEL AND ROGERS, 2011.
Different approaches can be followed to do so, that are complementary. A par-
ticipatory process will consist in engaging a discussion about people’s assump-
tions of cause-and-effect, including the undesired consequences that can be
anticipated. Social Sciences can also be used either to build initial assumptions,
but also to make sense of the assumptions behind solutions that have been pro-
posed, or again to identify unexpected, but likely consequences.

💡 Given that policy design is an iterative process, detailing the theory of

change can help rethink problem definition, or even be the starting point for
it. Many policies are actually “non-designed” KADIO, DAGENAIS, AND RIDDE 2018 and an
evaluation that appraises its theory of change can be the starting point of the
policy design process. It should be noted though than in an evaluation, the
theory of change is most of the time reconstructed to reflect the expecta-
tions of stakeholders at the moment of the evaluation, rather than at the be-
ginning during its conception. If an initial theory of change exists, though,
highlighting the differences and explaining them can be precious in assessing
a policy.
 BIBLIOGRAPHIE
Archibald, Thomas (2019) “What’s the Problem Lipsky, Michael. (1981). “Street-Level Bureau-
Represented to Be? Problem Definition Critique cracy: Dilemmas of the Individual in Public
as a Tool for Evaluative Thinking.” American Jour- Services.” Michigan Law Review 79 (4): 811.
nal of Evaluation, February, 109821401882404.
Monnier, Eric, Conan, Michel, Allen, Barbara,
Archibald, Thomas, Sharrock Guy, Buckley Jane, Duran, Patrice, Spenlehauer, Vincent, and
and Cook Natalie. (2016). “Assumptions, Con- Toulemonde, Jacques. (1992). Evaluations de
jectures, and Other Miracles: The Application of L’action Des Pouvoirs Publics.
Evaluative Thinking to Theory of Change Models
Morell, J. A. (2005). Why Are There Unintended
in Community Development.” Evaluation and
Consequences of Program Action, and What Are
Program Planning 59 (December): 119–27.
the Implications for Doing Evaluation? American
Astbury, Brad, and Frans L. Leeuw. (2010). Journal of Evaluation, 26(4), 444-463.
“Unpacking Black Boxes: Mechanisms and Theo-
ry Building in Evaluation.” American Journal of Pawson, Ray, and Nick Tilley. (2004). “Realist
Evaluation 31 (3): 363–81. Evaluation.” In Implementation Science, 3. BioMed
Central.
Bacchi, C. (2009). Analysing policy: What’s the
problem represented to be? Frenchs Forest, Rezsöhazy, R. (1996). Pour comprendre l’action
Australia: Pearson Education. et le changement politiques. De Boeck
Supérieur.
Bardach, E. (1977). The Implementation Game:
What Happens after a Bill Becomes a Law. Rittel, H., & Webber, M. (1973). Dilemmas in a
Cambridge: MIT Press. general theory of planning. Policy Sciences, 2,
155-169.
Bardach, E. (1980). ‘Implementation Studies
and the Study of Implements’. American Politi- Rossi, P. H., Lipsey, M. W., & Freeman, H. E.
cal Science Association. Berkeley: Graduate (2004). Evaluation: A systematic approach (7th
School of Public Policy University of California. ed.). Thousand Oaks, CA: Sage.

Connell, J.P, and Kubisch A.C. (1998). “Apply- Somerville, Peter. (2019). “A Critique of Cli-
ing a Theory of Change Approach to the Evalua- mate Change Mitigation Policy.” Policy & Politics.
tion of Comprehensive Community Initiatives: Weiss, Carol H. 1995. “Nothing as Practical as
Progress, Prospects, and Problems,” 16. Good Theory: Exploring Theory-Based Evalua-
Dye, T. R. (1972). Understanding Public Policy. tion for Comprehensive Community Initiatives
Englewood Cliffs, NJ: Prentice-Hall. for Children and Families.”

Funnell, S. C., & Rogers, P. J. (2011). Purpose- Wildavski, Aaron. 1979. “Speaking Truth to
ful program theory. Jossey-Bass. Power. The Art and Craft of Policy Analysis.

Howlett, Michael. (2010). Designing Public Poli-

cies: Principles and Instruments. 1st ed. Routledge.
Kadio, K, Dagenais C., and Ridde, V. (2018).
“De La Formulation d’une Politique Nationale à
La Compilation d’actions de Protection Sociale :
Un Cas de Non-Design Au Burkina Faso,” De-
cember, 1–22.
Kingdon, J. W. (1995). Agendas, Alternatives
and Publics Policies (2nd ed.). New York: Harp-
er Collins.
Lasswell, H. D. (1958). Politics: Who Gets
What, When, How. New York: Meridian.

To learn more about evaluation

Find guides and tools on evaluation on our website http://quadrant-conseil.fr/ressources.php

Re-use is possible by citing source: « Quadrant Conseil, 2020 »