You are here

Evaluation Research Methods
Share

Evaluation Research Methods

Four Volume Set
Edited by:
  • Elliot Stern - FAcSS, Emeritus Professor of Evaluation Research, University of Lancaster, UK


December 2004 | 1 736 pages | SAGE Publications Ltd
Evaluation is a set of research methods and associated methodologies with a distinctive purpose. They provide a means to judge actions and activities in terms of values, criteria and standards. At the same time evaluation is also a practice that seeks to enhance effectiveness in the public sphere and policy making. In order to improve as well as judge, there is a need to explain what happens and would have to be done differently for different outcomes to be achieved. It is in this explanatory mode that evaluation overlaps most directly with mainstream social science.

This collection offers a complete guide to evaluations research methods. It is organized in four volumes.

Volume 1 focuses on foundation issues and includes sections on the rationale for evaluation, central methodological debates, the role of theory and applying values, criteria and standards.

Volume 2 examines explaining through evaluation and covers sections on experimentation and causal inference, outcomes and inputs, socio-economic indicators, economics and cost benefit approaches and realist methods.

Volume 3 addresses qualitative methods and includes sections on case studies, responsive, developmental and accompanying evaluation, participation and empowerment, constructivism and postmodernism and multi-criteria and classificatory methods.

Volume 4 concentrates on evaluation to improve policy with sections on performance management, systematic reviews, institutionalization and utilization and policy learning and design.

The collection offers a unique and unparalleled guide to this rapidly expanding research method. It demonstrates how method and theory are applied in policy and strategy and will be an invaluable addition to any social science library.

Elliot Stern is the editor of Evaluation: the International Journal of Theory, Research and Practice, and works both as an independent consultant. He was previously Principal Advisor for evaluation studies at the Tavistock Institute, London.

 
VOLUME ONE
 
PART ONE: FOUNDATIONS ISSUES IN EVALUATION
 
SECTION ONE: TYPOLOGIES AND PARADIGMS
Michael Scriven
The Logic of Evaluation and Evaluation Practice
George Julnes and Melvin M Mark
Evaluation as Sensemaking
Knowledge Construction in a Realist World

 
Deborah M Fournier
Establishing Evaluative Conclusions
A Distinction between General and Working Logic

 
Thomas A Schwandt
Evaluation as Practical Hermeneutics
Ray Pawson and Nick Tilley
In with the New
Introducing Scientific Realism

 
Michel Demarteau
A Theoretical Framework and Grid for Analysis of Programme-Evaluation Practices
Peter van der Knaap
Policy Evaluation and Learning
Feedback, Enlightenment or Argumentation?

 
Evert Vedung
Merit Criteria and Performance Standards
Tineke A Abma and Mirko Noordegraaf
Public Managers amidst Ambiguity
Towards a Typology of Evaluative Practices in Public Management

 
 
SECTION TWO: ETHICS AND VALUES
Jennifer C Greene
Evaluation as Advocacy
Saville Kushner
How We Inhabit our Evaluations
Michael Morris
Research on Evaluation Ethics
What Have we Learned and Why Is It Important?

 
Clem Adelman
Anything Goes
Evaluation and Relativism

 
Saville Kushner
The Limits of Constructivism in Evaluation
Dianna L Newman
The Future of Ethics in Evaluation
Developing the Dialogue

 
 
SECTION THREE: QUANTITATIVE AND QUALITATIVE DEBATES
Melvin M Mark, Gary T Henry and George Julnes
Specifying Inquiry Modes
William R Shadish
Philosophy of Science and the Quantitative - Qualitative Debates
Thirteen Common Errors

 
Dawn Snape and Liz Spencer
The Foundations of Qualitative Research
 
VOLUME TWO
 
PART TWO: EVALUATION PRACTICE FOR ENGAGEMENT
 
SECTION ONE: CASE STUDIES AND DIALOGICAL METHODS
Robert K Yin
The Abridged Version of Case Study Research
Design and Method

 
Jessica Dart and Rick Davies
A Dialogical, Story-Based Evaluation Tool
The Most Significant Change Technique

 
Tineke A Abma and Robert E Stake
Stake's Responsive Evaluation
Core Ideas and Evolution

 
 
SECTION TWO: PARTICIPATION AND EMPOWERMENT
Ernest R House and Kenneth R Howe
Deliberative Democratic Evaluation in Practice
David M Fetterman
Empowerment Evaluation
Laura Nichols
Participatory Program Planning
Including Program Participants and Evaluators

 
Beatrice Floc'hlay and Eric Plottu
Democratic Evaluation
From Empowerment Evaluation to Public Decision-Making

 
Danielle Papineau and Margaret C Kiely
Participatory Evaluation in a Community Organization: Fostering Stakeholder Empowerment and Utilization
 
SECTION THREE: COMBINING METHODS AND DATA
Jennifer C Greene, Valerie J Caracelli and Wendy F Graham
Toward a Conceptual Framework for Mixed-Method Evaluation Designs
Mansoor A F Kazi
An Example of Qualitative Research Methods in Realist Evaluation
Daniel J McLinden and William MK Trochim
From Puzzles to Problems
Assessing the Value of Education in a Business Context with Concept Mapping and Pattern Matching

 
Jeff Hearn, John Lawler and George Dowswell
Qualitative Evaluations, Combined Methods and Key Challenges
General Lessons from the Qualitative Evaluation of Community Intervention in Stroke Rehabilitation

 
 
SECTION FOUR: PLANNING AND MANAGING EVALUATION
Murray Saunders
Beginning an Evaluation with RUFDATA
Theorizing a Practical Approach to Evaluation Planning

 
M F Smith
Evaluability Assessment
Reflections on the Process Method

 
Wolfgang Beywl and Philip Potter
RENOMO - A Design Tool for Evaluations
Designing Evaluations Responsive to Stakeholders' Interests by Working with Nominal Groups using the Moderation

 
 
SECTION FIVE: MULTI-SITE AND MULTI-LEVEL EVALUATION
Roger B Straw and James M Herrell
A Framework for Understanding and Improving Multisite Evaluations
Blaine R Worthen and Constance C Schmitz
Conceptual Challenges Confronting Cluster Evaluation
Colin Biott and Tina Cook
Local Evaluation in a National Early Years Excellence Centres Pilot Programme: Integrating Performance Management and Participatory Evaluation
Frances Lawrenz, Nanette Keiser and Bethann Lavoie
Evaluative Site Visits
A Methodological Review

 
 
VOLUME THREE
 
PART THREE: EXPLANATORY AND ECONOMIC MODELS
 
SECTION ONE: PROGRAMME THEORY AND LOGIC MODELS
Patricia J Rogers
Program Theory
Not Whether Programs Work but How They Work

 
Stewart I Donaldson
Theory-Driven Program Evaluation in the New Millennium
John A McLaughlin and Gretchen B Jordan
Logic Models
A Tool for Telling Your Programs Performance Story

 
Sharon Milligan et al
Implementing a Theory of Change Evaluation in the Cleveland Community-Building Initiative
A Case Study

 
Beth C Weitzman, Diana Silver and Keri-Nicole Dillman
Integrating a Comparison Group Design into a Theory of Change Evaluation
The Case of the Urban Health Initiative

 
Jules M Marquart
A Pattern-Matching Approach to Link Program Theory and Evaluation Data
Frans L Leeuw
Reconstructing Program Theories: Methods Available and Problems to be Solved
 
SECTION TWO: EXPERIMENTS AND THEIR LIMITS
Donald T Campbell
Reforms as Experiments
Thomas D Cook
The False Choice between Theory-Based Evaluation and Experimentation
Donald T Campbell
Methods for the Experimenting Society
Lone Scocozza
The Randomised Trial
A Critique from the Philosophy of Science

 
Jacques Toulemonde
Should Evaluation be Freed from Its Causal Links? An Answer Illustrated by European Economic Development Policies
 
SECTION THREE: ECONOMICS AND COSTS
Tom A J Sefton
Economic Evaluation in the Social Welfare Field
Making Ends Meet

 
Henry M Levin and Patrick J McEwan
Cost-Analysis Approaches in Evaluation and Decision Making
James J McEldowney
Policy Evaluation and the Concepts of Deadweight and Additionality
A Commentary

 
Ernest R House
The Limits of Cost Benefit Evaluation
Steven D Pinkerton et al
Ethical Issues in Cost-Effectiveness Analysis
 
PART FOUR: STRENGTHENING POLICIES AND PROGRAMMES
 
SECTION ONE: PERFORMANCE MANAGEMENT AND OUTCOMES
Ann Bonar Blalock
Evaluation Research and the Performance Management Movement
From Estrangement to Useful Integration?

 
Lawrence B Mohr
Outcomes and the Problem
 
VOLUME FOUR
 
PART FOUR: STRENGTHENING POLICIES AND PROGRAMMES (continued)
 
SECTION ONE: PERFORMANCE MANAGEMENT AND OUTCOMES (continued)
Ingrid M E Munck
Bridging the Macro and Micro Levels in Outcome Evaluation
Frans-Bauke van der Meer
Evaluation and the Social Construction of Impacts
Keith Tones
Effectiveness in Health Promotion
Indicators and Evidence of Success

 
Nathalie Holvoet and Robrecht Renard
Desk Screening of Development Projects
Is It Effective?

 
Burt Perrin
Effective Use and Misuse of Performance Measurement
M A Scheirer and K Newcomer
Opportunities for Program Evaluators to Facilitate Performance-Based Management
 
SECTION TWO: CAPACITY AND UTILISATION
Arnold J Love
Developing Systematic Internal Evaluation
Rosalie T Torres
Improving the Quality of Internal Evaluation
The Evaluator as Consultant-Mediator

 
Jonathan A Morell
Internal Evaluation
A Synthesis of Traditional Methods and Industrial Engineering

 
Carol H Weiss
Have We Learned Anything New about the Use of Evaluation?
Michael Quinn Patton
The Evaluator's Responsibility for Utilization
Richard C Sonnichsen
Building Evaluation Capacity within Organisations
Ville Valovirta
Evaluation Utilization as Argumentation
 
SECTION THREE: POLICY, EVIDENCE AND LEARNING
Carol H Weiss
Evaluation for Decisions
Is Anybody There? Does Anybody Care?

 
Sandra Nutley, Isabel Walter and Huw T O Davies
From Knowing to Doing
A Framework for Understanding the Evidence-Into-Practice Agenda

 
Ray Pawson
Evidence-Based Policy
In Search of a Method

 
Ray Pawson
Evidence-Based Policy
The Promise of `Realist Synthesis'

 
Paul A Sabatier
Top-Down and Bottom-Up Approaches to Implementation Research
Sidsel Sverdrup
Towards an Evaluation of the Effects of Laws
Utilizing Time-Series Data of Complaints