Impact Evaluation Of Development Interventions

Impact evaluation is an empirical approach to estimating the causal effects of interventions, in terms of both magnitude and statistical significance. Expanded use of impact evaluation techniques is critical to rigorously derive knowledge from development operations and for development investments and policies to become more evidence-based and effective. To help backstop more use of impact evaluation approaches, this book introduces core concepts, methods, and considerations for planning, designing, managing, and implementing impact evaluation, supplemented by examples. The topics covered range from impact evaluation purposes to basic principles, specific methodologies, and guidance on field implementation. It has materials for a range of audiences, from those who are interested in understanding evidence on "what works" in development, to those who will contribute to expanding the evidence base as applied researchers.

This book offers guidance on the principles, methods, and practice of impact evaluation. It contains material for a range of audiences, from those who may use or manage impact evaluations to applied researchers. Impact evaluation is an empirical approach to estimating the causal effects of interventions, in terms of both magnitude and statistical significance. Expanded use of impact evaluation techniques is critical to rigorously derive knowledge from development operations and for development investments and policies to become more evidence-based and effective. To help backstop more use of impact evaluation approaches, this book introduces core concepts, methods, and considerations for planning, designing, managing, and implementing impact evaluation, supplemented by examples. The topics covered range from impact evaluation purposes to basic principles, specific methodologies, and guidance on field implementation. It has materials for a range of audiences, from those who are interested in understanding evidence on "what works" in development, to those who will contribute to expanding the evidence base as applied researchers.

In recent years debates on as well as funding of impact evaluations of development interventions have flourished. Unfortunately, controversy regarding the promotion and application of randomized experiments (RE) has led to a sense of polarization in the development policy and evaluation community. As some proponents claim epistemological supremacy of REs (with respect to attribution) the counter reaction among others has been rejection. Needless to say, such extreme positions are counterproductive to reaching a goal that is commonly endorsed: to learn more about what works and why in development. This paper discusses the prospects and limitations of REs from the perspective of three categories of challenges in impact evaluation: delimitation and scope, attribution versus explanation, and implementation challenges. The implicit lesson is twofold. First of all, the question 'to randomize or not to randomize' is overrated in the current debate. Limitations in scope, applicability as well as implementation will necessarily restrict the use of REs in development impact evaluation. There is a risk that the current popularity of REs in certain research and policy circles might lead to a backlash as too high expectations of REs may quicken its demise. More importantly, given the nature and scope of the challenges discussed in the paper, more energy should be devoted to developing and testing 'rigorous' mixed method approaches within a framework of theory-driven evaluation.

The second edition of the Impact Evaluation in Practice handbook is a comprehensive and accessible introduction to impact evaluation for policy makers and development practitioners. First published in 2011, it has been used widely across the development and academic communities. The book incorporates real-world examples to present practical guidelines for designing and implementing impact evaluations. Readers will gain an understanding of impact evaluations and the best ways to use them to design evidence-based policies and programs. The updated version covers the newest techniques for evaluating programs and includes state-of-the-art implementation advice, as well as an expanded set of examples and case studies that draw on recent development challenges. It also includes new material on research ethics and partnerships to conduct impact evaluation. The handbook is divided into four sections: Part One discusses what to evaluate and why; Part Two presents the main impact evaluation methods; Part Three addresses how to manage impact evaluations; Part Four reviews impact evaluation sampling and data collection. Case studies illustrate different applications of impact evaluations. The book links to complementary instructional material available online, including an applied case as well as questions and answers. The updated second edition will be a valuable resource for the international development community, universities, and policy makers looking to build better evidence around what works in development.

Impact evaluation aims to answer whether and to what extent a development intervention has delivered its intended effects, thus enabling evidence-based policy making. The desire for more hard evidence of the effectiveness of development interventions has fueled a growing interest in rigorous impact evaluation in the international development community. This report discusses the fundamental challenge of impact evaluation, which is to credibly attribute the impact, if any, to the intervention concerned. It then discusses the merits and limitations of various impact evaluation methods. It also presents a survey of recent applications of impact evaluation, focusing on the typical evaluation problems looked at, methods used, and key findings. The report includes six case studies and outlines practical steps in implementing an impact evaluation.

Despite the billions of dollars spent on development assistance each year, there is still very little known about the actual impact of projects on the poor. There is broad evidence on the benefits of economic growth, investments in human capital, and the provision of safety nets for the poor. But for a specific program or project in a given country, is the intervention producing the intended benefits and what was the overall impact on the population? Could the program or project be better designed to achieve the intended outcomes? Are resources being spent efficiently? These are the types of questions that can only be answered through an impact evaluation, an approach which measures the outcomes of a program intervention in isolation of other possible factors. This handbook seeks to provide project managers and policy analysts with the tools needed for evaluating project impact. It is aimed at readers with a general knowledge of statistics. For some of the more in-depth statistical methods discussed, the reader is referred to the technical literature on the topic. Chapter 1 presents an overview of concepts and methods. Chapter 2 discusses key steps and related issues to consider in implementation. Chapter 3 illustrates various analytical techniques through a case study. Chapter 4 includes a discussion of lessons learned from a rich set of "good practice" evaluations of poverty projects which have been reviewed for this handbook.

Recognizing that complexity calls for innovative, conceptual, and methodological solutions, Dealing with Complexity in Development Evaluation by Michael Bamberger, Jos Vaessen, and Estelle Raimondo offers practical guidance to policymakers, managers, and evaluation practitioners on how to design and implement complexity-responsive evaluations that can be undertaken in the real world of time, budget, data, and political constraints. Introductory chapters present comprehensive, non-technical overviews of the most common evaluation tools and methodologies, and additional content addresses more cutting-edge material. The book also includes six case study chapters to illustrate examples of various evaluation contexts from around the world.

Assessing impacts of public investments has long captured the interest and attention of the development community. This paper presents the evolution of different methods and approaches used for ex ante appraisal, monitoring, project evaluation, and impact assessment over the last five decades. Among these tools, impact assessment (IA) conducted retrospectively comes closest to providing the proof of development effectiveness. It is defined as the systematic analysis of the significant or lasting changes in people's lives brought about by a given action or series of actions in relation to a counterfactual. There are three basic types of retrospective IAs: macro-level IAs that focus on the contribution of developmental efforts to an impact goal aggregated at a sector or a system level; micro-level impact evaluations (IEs) concerned with estimating the average effect of an intervention on outcomes at the beneficiary level; and micro-level ex post impact analysis concerned with total effects of a development effort after the outputs are scaled-up. Ex post IAs have evolved and expanded over the decades in both breadth and depth of analysis in response to evolving development themes and methodological advancements. The increased emphasis on learning from evaluations has also seen responses from both quantitative and qualitative camps of the evaluation community. The paper argues that generation of robust knowledge that feeds into making developmental policies and investment decisions requires a hierarchical and cumulative approach to "improving the proof" through rigorous and a variety of impact assessment methods applied incrementally at the project, program and system level. Subjecting as many development interventions as resources allow to rigorous impact assessment based on a common framework can help build a critical body of evidence on impacts of development interventions, which can then be subjected to meta-analyses to help assimilate results across different studies and build a knowledge base on what works and what does not.