This document preceeds an IEG evaluation and provides information on the key questions, rationale, scope, and evaluation approach. Annual Review of Development Effectiveness These annual reviews are a meta-evaluation that provide a comprehensive assessment of the Bank's development effectiveness. They offer a broad assessment of the Bank's contribution to development effectiveness, drawing on project evaluations and IEG sector, thematic and country reviews.
Programmatic partnerships in which the partners dedicate resources, the activities of the program are global, regional or multicountry in scope, and the partners establish a new organization with shared governance and a management unit to deliver activities.
Resources that offer access to key findings, recommendations, lessons, and knowledge-products drawn from the World Bank Group's development experiences. These include lesson notes, briefs, summaries, cluster reports and working papers.
Completed projects are evaluated through Project Performance Assessment Reports (PPARs) and Implementation Completion Report (ICR) Reviews. They are carried out after Bank funds have been fully disbursed to a project, are similar to the completion evaluations carried out by many agencies. (to search ICR Reviews, see "Project Completion Validations")
Thematic Reviews examine Bank performance and experience in a lending sector (such as agriculture, transport, and the like) or thematic area (poverty, gender, and so forth) over five to ten years and report on their conformity to Bank policy and good practice, as well as on the development effectiveness of the Bank's activities.
Embrace all APPROPRIATE evaluation methods.
The challenge with inclusiveness is that it can lead to a watered-down inability to make any useful conclusions: When all methods are equally valid, we are left with no way to discern between competing claims. Rather, we should endeavor to apply the right method to the right question. In doing so we must be clear about the explicit identification assumptions as well as the too-often implicit ontological and epistemological assumptions behind different methods of evaluation. Although there is no hierarchy of evaluation methods generally, there is for specific questions. So while an experimental or quasi-experimental design is probably not the right method to answer the question about how stakeholders viewed the rollout process of a particular type of intervention, it is the right method to answer the question of the causal, attributable effect of a defined intervention in raising a defined construct of welfare for a defined population at a particular place and time.
The question of external validity is a valid one. As Lant points out, there are few “invariance laws” for social science. In practice, therefore, the general principle of “external validity” is not likely to be accurate—there is no such thing as general generalizability in social sciences. Moreover, the question of out of sample prediction, which is what critics usually mean by external validity, is not limited to any particular type of evaluation method; all have the same challenge. Rather, we must have a particular, defined context in mind when discussing the transferability of programmatic findings, regardless of the evaluation method that generated those findings. The advantage of systematic reviews is not that they prescribe universal policy, but that they present those interventions that tend to be robust to the many ways things can go wrong and describe the contexts in which they worked to allow policy makers to form their own judgments on the appropriateness of local application.
Beyond the oft-repeated and as often agreed to bit of useless wisdom that “context matters”, the challenge for evaluation—of all stripes—moving forward will be understanding which are the important elements of context that matter. While the applicability of a finding from one intervention to another place, time, or scale is never 100%, neither is it zero. Discovering the key elements upon which transferability is likely to hinge may well rely on both social science theory and the arsenal of evaluation methods, appropriately applied.