93  

  Journal  of  MultiDisciplinary  Evaluation  

Book  Review:  Evaluation  Theory,  

Volume  11,  Issue  25,  2015  

Models  &  Applications    

(2014)  2nd  Edition.  Daniel  L.  Stufflebeam  and  Chris  L.    
ISSN  1556-­‐8180  
S.  Coryn.  San  Francisco:  Jossey-­‐Bass.   http://www.jmde.com  
 
Cheryl  L.  Endres  
Western  Michigan  University  
 
Erica  L.  Fiekowsky  
Western  Michigan  University  
 
Sabrina  Holley  
Western  Michigan  University  
 

Background:  N/A   Intervention:  N/A  

   
Purpose:  Review  of  the  updated  version  of  Evaluation  theory,   Research  Design:  N/A  
methods   &   applications,   previously   coauthored   by   D.   L.    
Stufflebeam  and  A.  J.  Shinkfield.   Data  Collection  and  Analysis:  N/A  
   
Setting:  N/A   Findings:  N/A  
   
Keywords:  book  review;  revised;  evaluation  approach;  handbook.  

   

 
94           Endres,  Fiekowsky,  &  Holley  

 
The authors’ intended use for Evaluation Theory, ranging from pseudoevaluations to eclectic
Methods, & Applications, as stated in the Preface, evaluations. Each chapter begins either with a
is “as a textbook for graduate-level courses in background and introduction to the category (for
program evaluation and as a tool for evaluators the Pseudoevaluations), a definition of the
and administrators and other clients and users of category (for the quasi-evaluation studies and
evaluation” (p. xix). Graduate students learning improvement-and accountability oriented
about evaluation will certainly appreciate this approaches) or an overview of the approaches (for
second edition’s use as a textbook, due to its the social agenda and advocacy evaluation and
comprehensive nature. The book provides eclectic approaches). Each of the approaches is
information on the evaluation field’s history, described, with a section on the general strengths
theories, and standards; various approaches to and weaknesses of each of the approaches within
evaluation; select qualitative and quantitative the chapter provided. Part 2 is pulled together in
methods; and steps and checklists helpful to the Chapter 10. In this chapter, the authors discuss the
implementation of evaluations. The book may be rationale used to select the approaches included,
less useful to experienced professionals, who may describe the methodology used, their
be more likely to already have the generalized qualifications and conflicts of interest. Standards
knowledge covered in this text. For those already used in the text for judging the featured evaluation
possessing some degree of familiarity with approaches are discussed.
evaluation theory, Evaluation Theory, Methods & Part Three explains in great detail six of the
Applications may serve as a handbook, or tool, for approaches mentioned in the previous section. The
conducting evaluations, both in terms of approach first two, Chapters 11 and 12, elaborate on quasi-
selection and application of the checklists. evaluations, the first of which is experimental and
Stufflebeam and Coryn’s contribution to the quasi-experimental design, including applicability,
growing field of evaluation texts succeeds examples of misapplication, and alternative
admirably in fulfilling their intended purpose to approaches and contexts. The other quasi-
provide “one significant and useful set of insights evaluation approach discussed is the case study
into the complex sphere of program evaluation” (p. evaluation. This chapter concentrates on case
xix). studies as an application of a larger program
evaluation. Chapter 13 is devoted to Stufflebeam’s
Organization CIPP model. There are several tables included in
this chapter to help summarize the different
aspects of the CIPP model, including one for
The book is organized into five parts. Below,
illustrative questions, objectives, methods, and
we provide a brief overview of each part, followed
uses of the four types of evaluation, and the CIPP
by a critique of the text using the lenses of doctoral
model’s use as a systems strategy for
students, new to the field of evaluation.
improvement. Additionally, an extended
With the multiple intended audiences in mind,
hypothetical example is given to help explain the
the authors begin by providing context and
social systems approach. Chapter 14, titled the
background to create a framework for the rest of
“consumer-oriented approach,” touches briefly on
the text. Part One lays this groundwork for the rest
this approach attributed to Michael Scriven. The
of the book and contains three chapters covering
larger part of the chapter focuses on Scriven’s
the foundational underpinnings of evaluation in
many contributions to the transdiscipline of
terms of history, purpose, theory, and standards.
evaluation. These contributions include concepts
Part Two contains seven chapters, which cover
as well as approaches. In the final two chapters in
23 evaluation approaches grouped by the authors
this section, the authors expound on the
into categories. The first chapter in this section
responsive or stakeholder-centered approach and
explains the importance of understanding that
the utilization-focused evaluation approach. The
various approaches can be used and that selection
stakeholder-centered approach chapter includes
should be driven by factors including the primary
an example of its application. While the chapter on
purpose for the evaluation. Chapter 4 provides
utilization-focused evaluation does not contain a
detail on the strategy the authors use to review and
similar illustration of the approach, it does include
classify the 23 evaluation approaches contained in
a step-by-step guide on how to go about carrying
chapters five through nine. Description of other
out the evaluation.
classification schemes, and a rationale for the
procedures used in the classification scheme
within the text is provided. The next five chapters
are devoted to the categories of approaches,

 
Journal  of  MultiDisciplinary  Evaluation     95    

 
Part Four focuses on practical evaluation that concisely summarize detailed information,
tasks, procedures, and tools. This section begins which seems improved from the previous edition.
with the basics: How does one secure an The text did leave us wondering about a few
evaluation contract? Once one has successfully weaknesses. For example, in the eclectic
won a contract, how does one decide whom to hire approaches section, it is not clear why some of the
for their evaluation team? Then, the section moves approaches were included. In the introduction to
into other practical aspects of evaluation like the book, historical importance and prevalence in
budgeting, contracting, data collection, and the field are mentioned, but the rationale is not
reporting. The final chapters round out the nuts explained for each approach. In terms of current
and bolts of practice, and discuss data collection, topical coverage, there is no discussion of
analysis and synthesis as well as communication of complexity or systems thinking, and very little
findings. discussion of empowerment evaluation other than
Part Five includes two chapters that discuss as a quasi-evaluation approach. The text
the need for metaevaluation (Chapter 25) to emphasizes the interplay between theory and
improve evaluation practice and the rationale and practice, which is a key issue in the field of
purpose for institutionalizing and mainstreaming evaluation, but it spends more time on theory than
evaluation. Both of these topics are very timely, practice. We found that the limitations and
given that 2015 has been declared the examples for each approach were particularly
International Year of Evaluation by EvalPartners, illuminating in understanding how and when each
which includes many prominent evaluation approach should be used. However, in discussing
groups, such as the American Evaluation evaluation approaches with other students and
Association, European Evaluation Association, practitioners, it became clear that the theoretical
International Organization for Evaluation approaches are not often used in practice, and this
Cooperation, and the United Nations Evaluation issue was not adequately addressed in the text.
Group, among numerous others. Chapter 26 Most of the book was consistent in formatting,
provides a summary discussion of the nine themes structure, and level of detail. Part One was an
of the text and offers practical steps that effective introduction to evaluation, Part Two
organizations can use to make evaluation an presented different approaches in a handbook-
integrated practice within organizations, as style format, and Part Four continued the
opposed to an occasional, or required task. In handbook-style with practical applications of
addition to providing an overview of growth in the conducting an evaluation. Part Three did not fit
field that contributes to the possibility of making well with the flow of the rest of the book. While
this a reality in the near future, a checklist is considered an extension, or elaboration, of select,
included that organizations can use to guide their previously-mentioned approaches, Part Three was
efforts. more a conglomeration of historic perspective,
multiple perspectives (theories) on a single
Reviewers’  Opinions approach, and significant contributions to the
transdiscipline of evaluation.
Evaluation Theory, Methods, & Applications
provides a comprehensive introduction to Addressing  Previous  Reviews
evaluation as a field—so much so that it should be
used for a two-semester course rather than one- Reviews of the first edition of this text had several
semester. The book is beneficial to students as a criticisms that were addressed in the new edition.
textbook, although evaluation practitioners would Most notably, one review (Wiersma, 2009) found
also benefit from the checklists, how-to guides, the text to have too much documentation and not
and summaries and components of approaches. enough figures. The second edition includes
Among its strengths for classroom use are the various types of tools and visual aids—including
detailed learning objectives at the beginning of figures, tables, checklists, and exhibits—to support
chapters and the summaries and review activities the wealth of information in the text.
at the end to organize thinking. These are helpful Another criticism of the first edition was that
both for students as a focus to chapter readings most supporting examples came from the field of
and instructors as a source for learning education (Wallis, 2008). While education
assessment. Descriptions of approaches and remains prominent in the second edition, the
theories are made relevant and applicable to real variety of examples offered—including health,
world uses and contexts throughout the text. Of public administration, and education—was
particular use to students are tables throughout sufficient to us. Wallis did not think that the

 
96           Endres,  Fiekowsky,  &  Holley  

 
authors had sufficiently differentiated between References  
approaches, models, theories, and applications. In
the current edition, the authors briefly mention
Datta, L. (2007). [Review of the book: Evaluation
their rationale for their choice of terms, but
Theory, Models, and Applications, by D.L.
additional clarification would be useful. Wallis also
Stufflebeam & A.J. Shinkfield]. American
thought that the authors left the difference
Journal of Evaluation. Vol. 28 (4). doi:
between program evaluation and evaluation
10.1177/1098214007308902.
research vague, but the definition of evaluation as
Wallis A, Barnette J. (2008). [Review of the book:
a transdiscipline in the current edition adds some
Evaluation Theory, Models, and
clarity.
Applications, by D.L. Stufflebeam & A.J.
Datta’s (2007) review of the first edition took
Shinkfield]. JAMA.2008;299(22):2692-
issue with the lack of detail included about certain
2693. doi:10.1001/jama.299.22.2692.
topics, in particular empowerment evaluation,
Wiersma, W. (2009). [Review of the book:
systems theories, and complex adaptive systems.
Evaluation Theory, Models, and
The authors do give attention to empowerment
Applications, by D.L. Stufflebeam & A.J.
evaluation, but only as a pseudoevaluation
Shinkfield]. Journal of MultiDisciplinary
approach, which some readers may find
Evaluation. Vol. 6 (11).
problematic due to its prominence in the field.
However, the authors provide justification for
their classification, by pointing out that once
evaluators take on the role of social advocates or
shift their focus to empowering a population of
interest, bias enters into the evaluation. In
addition, the degree of control that is given to the
client for the evaluation often means that the
evaluator is lending his or her name more than his
or her expertise to the final evaluation product.
While some attention to empowerment evaluation
was included in this text, systems theories and
complex adaptive systems continue to be omitted
from the second edition.

Ideas  for  Future  Editions

While it may be beyond the scope of any single text
to cover all possible applications of evaluation, a
brief discussion of emergent trends could be
incorporated, with reference to external sources
for more information. Today, that would include
systems theory and complex adaptive systems. By
the time the next edition is published, these may
be more established, or may have evolved, in
which case the upcoming trends would be
different. An alternative would be to include a
section on the political nature of evaluation, and
how approaches and practice are influenced by
context, which includes politics.