Organization
World Bank
Report Year
2015
1st MAR Year
2017
Accepted
Yes
Status
Active
Recommendation

Expand voluntary evaluations that respond to learning needs of management and teams. These include impact and process evaluations, retrospectives, and beneficiary surveys and need not be project-specific but can cover multiple interventions in a given sector, country, or region, depending on learning needs. Building on recent progress, further enhance the manner in which impact evaluations respond to learning needs through greater regional and thematic selectivity and enhance the uptake of findings from impact evaluations. Ensure that information technology systems capture and make accessible knowledge from self-evaluations (applies to the World Bank and IFC).abc

Recommendation Adoption
IEG Rating by Year: mar-rating-popup N NT NT NT Management Rating by Year: mar-rating-mng-popup M NT NT NT
CComplete
HHigh
SSubstantial
MModerate
NNegligible
NANot Accepted
NRNot Rated
Findings Conclusions

Corporate requirements specify the scope, timing, and content of self-evaluations in a way that supports reporting more than it does learning. For example, most self-evaluations continue to be project-specific, with similar approach and depth, regardless of the learning potential. Mandatory and voluntary self-evaluations are not used strategically to meet knowledge gaps and approaches to using them for lesson learning are fragmented, further fueling staff perceptions of low importance. There is scope to strengthen Bank-wide oversight and the regional and thematic selectivity of impact evaluations, the uptake of findings from impact evaluations, and the use of information systems for capturing, classification, and availability of Bank Group mandatory and voluntary self-evaluations. IFC has a fragmented approach to lesson learning with no clear framework for capturing, storing and acting on lessons and no high-level champion for this has emerged.

Original Management Response

World Bank Management agrees that voluntary self-evaluations such as impact evaluations are an important way to assess effectiveness of the interventions. Making the existing IEs more easily accessible to staff through a centralized repository would be an important first step to promote the wider use of this wealth of information. While it is not feasible or necessary for every operation to have an embedded IE, there is a need for a more systematic approach and for guidance on IEs to boost learning and development effectiveness. Management will establish a working group to discuss options and a way forward.

IFC Management agrees with the recommendation that an evaluation program should be developed in response to knowledge gaps identified by IFC business specialists and operational teams or clients. IFC's Development Impact Unit has developed a framework of evaluations to (a) increase the relevance of evaluations for IFC and its stakeholders, (ii) improve insights and operational learning and (c) improve communication of evaluation findings. A platform is being built to give operational teams access to the most relevant and up-to-date lessons, and IFC is developing guidelines and training on how to prepare such lessons.

Action Plans
Action 1
Action 1 Number:
0391-01
Action 1 Title:
5(a): Impact evaluations (IEs)
Action 1 Plan:

WB Action 5.1: Establish a working group to explore options to conduct IEs in a more systematic manner with greater linkages to operations to address knowledge gaps. [FY17]
WB Action 5.2: Conduct a stocktaking of existing IEs and develop a central repository of impact evaluations that is regularly updated [FY18 onwards].
WB Action 5.3: Pilot an in-depth analysis of IEs in selected GPs on how the knowledge generated has been used to recalibrate operations or inform the design of new operations [FY18]
WB Action 5.4: Define and monitor a new corporate indicator on impact evaluations supported by the World Bank in IDA countries [FY18 onwards].

Action 2
Action 2 Number:
0391-02
Action 2 Title:
5(b): Other voluntary evaluations
Action 2 Plan:

WB Action 5.5: Building on the President’s commitment to include beneficiary feedback in all projects where beneficiaries are clearly identifiable, develop resources, including guides, how-to notes, and training opportunities, for task teams to collect beneficiary feedback in real time [FY17 onwards]

Action 3
Action 4
Action 5
Action 6
Action 7
Action 8
2020
IEG Update:
No Updates
Management Update:
No Updates
2019
IEG Update:
No Updates
Management Update:
No Updates
2018
IEG Update:
No Updates
Management Update:
No Updates
2017
IEG Update:

IEG notes that management has set up a working group on IEs and conducted a stocktake of past IEs, while also created a new corporate indicator. These activities are interesting yet still in their early stages. It is therefore too early to assess whether they will achieve the goal of this recommendation, which is to enhance the manner in which IEs respond to learning needs, enhance the regional and thematic selectivity of World Bank IEs, and enhance the use of findings from impact evaluations. The new corporate indicator on impact evaluations supported by the World Bank in IDA countries is interesting IEG hopes this will focus not only on total production of IEs but also on their use. In part, this use will depend on how well they cover gaps and respond to operational learning needs.

On 5(b), while it is true that a wealth of resources are available on citizen engagement and beneficiary surveys, it is crucial that such surveys be used to inform course corrections and operational self-evaluation.

Management Update:

This is an adaptable MAR, where progress is assessed against the outcome direction, and the actions are indicative for each fiscal year, and can be updated each year. For outcome direction 5a (IEs) the FY17 updates are as follows:WB Action 5.1: Working group has been established to explore options to conduct IEs in a more systematic manner with greater linkages to operations to address knowledge gaps. [FY17]WB Action 5.2: A stocktaking of existing IEs was conducted in FY17. Building on the IE stocktaking exercise, a discussion is under progress to develop a central repository of impact evaluations that is regularly updated [FY18 onwards].WB Action 5.3: Preparation is under progress to pilot an in-depth analysis of IEs in selected GPs on how the knowledge generated has been used to recalibrate operations or inform the design of new operations [FY19]WB Action 5.4: A new corporate indicator on impact evaluations supported by the World Bank in IDA countries was defined and will be monitored [FY18 onwards]For outcome direction 5b (other voluntary evaluations) the FY17 updates are as follows:WB Action 5.5: Building on the President's commitment to include beneficiary feedback in all projects where beneficiaries are clearly identifiable, the Citizen Engagement Secretariat has developed a detailed website that includes a range of resources, including guidance, how-to notes, tools and toolkits, training and learning resources, and case studies and other reflections on experiences with particular citizen engagement mechanisms. In addition, quarterly in-person and online clinics are available to staff on a quarterly basis, and recorded for use at any time. This can be accessed on the intranet with the shorthand www.citizengagement