Outcome Ratings Increase Across Entire World Bank Group
Web Resource
While the RAP 2021 presents encouraging results, as the impact of the pandemic continues to permeate Bank Group operations and outcome measurement systems, outcome ratings may decline in future reviews of the World Bank Group's development effectiveness.While the RAP 2021 presents encouraging results, as the impact of the pandemic continues to permeate Bank Group operations and outcome measurement systems, outcome ratings may decline in future reviews of the World Bank Group's development effectiveness.
Measuring Results in Global Development
Web Resource
We are less than a decade from delivering on the Sustainable Development Goals (SDGs). The SDGs touch on some of the most profound challenges facing humanity, such as ending poverty and reducing inequality. What does it mean to assess results on this massive agenda? Senior evaluator Estelle Raimondo and host Brenda Barbour talk about what they’ve learned about results measurement in global Show More
We are less than a decade from delivering on the Sustainable Development Goals (SDGs). The SDGs touch on some of the most profound challenges facing humanity, such as ending poverty and reducing inequality. What does it mean to assess results on this massive agenda? Senior evaluator Estelle Raimondo and host Brenda Barbour talk about what they’ve learned about results measurement in global development. How do we rethink Results Based Management to better support countries achieve their development outcomes? And what can the COVID-19 pandemic tell us about current results measurement practices?
Listen on Spotify, Apple Podcasts or Google Podcasts.
Related resources
Evaluation: The World Bank Group Outcome Orientation at the Country Level
Blog: The Results Agenda Needs a Steer – What Could Be its New Course?
Results and Performance of the World Bank Group 2021
Web Resource
This report, also known as the RAP, is the 2021 annual review of World Bank Group’s development effectiveness. The report assesses the Bank Group’s performance by analyzing the achievement of project and program objectives through validated ratings and presenting an in-depth analysis of possible drivers.
This report, also known as the RAP, is the 2021 annual review of World Bank Group’s development effectiveness. The report assesses the Bank Group’s performance by analyzing the achievement of project and program objectives through validated ratings and presenting an in-depth analysis of possible drivers.
Validation of Management Progress on IEG Evaluation Recommendations – An Independent Evaluation
PDF file
The Management Action Record (MAR) system tracks follow-up to IEG evaluations. This document validates management’s self-assessment covering FY17-21. The Management Action Record (MAR) system tracks follow-up to IEG evaluations. This document validates management’s self-assessment covering FY17-21.
Results and Performance of the World Bank Group 2021 – Concept Note
PDF file
The Results and Performance of the World Bank Group (RAP) report is the annual review of evidence from IEG evaluation and validation work on the development effectiveness of the World Bank Group (WBG). It will be the eleventh in a series that began in 2010; it will also be the second report departing from the exclusive traditional focus on ratings to also provide additional evidence on the Show MoreThe Results and Performance of the World Bank Group (RAP) report is the annual review of evidence from IEG evaluation and validation work on the development effectiveness of the World Bank Group (WBG). It will be the eleventh in a series that began in 2010; it will also be the second report departing from the exclusive traditional focus on ratings to also provide additional evidence on the nature of intended outcomes across the WBG. RAP 2021 will build and expand on the RAP 2020 innovations by refining the classification framework for intended outcomes and integrating analysis of existing ratings (trends) with the outcome classification analysis. Like past RAP reports, RAP 2021 will provide an analysis of project ratings and factors associated with performance as measured by those ratings. Expanding on the past, RAP 2021 will analyze and interpret these ratings through the lens created by the refined typology of intended outcomes. This lens would enable an examination of ratings that takes into account portfolio composition in terms of the type (classification) of intended outcomes, as well as the likelihood of achieving those intended outcomes. In other words, RAP 2021 aims at providing a joint assessment of ratings and the risk-return profile of the portfolio generating those ratings.
World Bank Operations Evaluation Department : the first 30 years
PDF file
During the course of 2002-03 the Bank’s Operations Evaluation Department (OED) held a
series of major events, including seminars and workshops, leading up to the thirtieth
anniversary of the founding of OED by Robert McNamara on July 1, 1973. This volume
captures the history of OED as related by many of those who participated in making that
history. It is divided into four sections.
During the course of 2002-03 the Bank’s Operations Evaluation Department (OED) held a
series of major events, including seminars and workshops, leading up to the thirtieth
anniversary of the founding of OED by Robert McNamara on July 1, 1973. This volume
captures the history of OED as related by many of those who participated in making that
history. It is divided into four sections.
Mobilizing Technology for Development
Web Resource
This evaluation assesses the preparedness of the World Bank and the International Finance Corporation to help clients harness the opportunities and mitigate the risks posed by disruptive and transformative technologies.
This evaluation assesses the preparedness of the World Bank and the International Finance Corporation to help clients harness the opportunities and mitigate the risks posed by disruptive and transformative technologies.
The Results Agenda Needs a Steer—What Could Be its New Course?
Web Resource
Is it time to rethink Results Based Management in international development? We imagine a system that favors learning over compliance.Is it time to rethink Results Based Management in international development? We imagine a system that favors learning over compliance.
What the World Bank Group’s Performance Results Cannot Tell Us About Development Outcomes
Web Resource
Many World Bank Group projects display strong performance. These results however tell us little about the Bank Group’s aggregate contributions and higher-level development outcomes—the results measurement systems are designed to tell a different type of performance story.
The World Bank Group’s (WBG) results measurement systems for projects, programs, and thematic areas are purposefully Show MoreMany World Bank Group projects display strong performance. These results however tell us little about the Bank Group’s aggregate contributions and higher-level development outcomes—the results measurement systems are designed to tell a different type of performance story.
The World Bank Group’s (WBG) results measurement systems for projects, programs, and thematic areas are purposefully designed.
Generally, they focus on collecting data on indicators that can be attributed to Bank Group interventions. They help verify whether targets have been met. They provide consistency in displaying results across portfolios. Importantly, they allow shareholders to hold the Bank Group accountable.
In our flagship report, IEG aggregates and analyzes much of this data to give the big picture on the Results and Performance of the World Bank Group (RAP 2020).
The 2020 RAP report finds that the World Bank and MIGA performs strongly overall, and that IFC’s decline in ratings that started ten years ago appears to have halted. We welcome you to dive deeper into how various arms and instruments of the Bank Group perform, and what factors help explain these trends.
At the same time, while being able to attribute results to Bank Group projects remains important for reasons of accountability, there is also a need to better capture how the institution contributes to lasting positive change in client countries.
We refer to this as higher-level development outcomes—social and economic change on the ground which Bank Group interventions cannot achieve singlehandedly, but which the institution can meaningfully add to, over time.
How well does the World Bank Group contribute to these sorts of development results? RAP 2020 finds that answering this question would require searching beyond the institution’s existing results measurement systems.
Limited evidence on higher-level outcomes is gathered
Following an analysis of the Bank Group’s results reporting—from projects to country programs to thematic priorities such as gender and climate change—the RAP 2020 arrived at the conclusion that the World Bank Group collects limited systematic evidence on its contribution to higher-level outcomes.
Higher-level outcomes stem from the interplay of different projects and types of Bank Group engagements—lending, knowledge, and convening—over time. The World Bank Group’s Board has requested more evidence on these sorts of development outcomes and how the interventions help achieve the Sustainable Development Goals (SDGs). Better evidence on higher level outcomes would also help with learning, reflections on strategy, and course corrections where needed.
Many of the World Bank Group’s existing corporate results measurement systems were designed to collect data needed for ratings and for process and compliance monitoring. These systems do what they were designed to do.
For example, individual projects’ self-evaluations mostly do a good job at capturing project-level results. IEG rates these self-evaluations, as part of a system that provides consistency in displaying results across sectors, across types of projects, and across time. Yet combining performance ratings from individual projects cannot give a clear picture of the important development outcomes to which the Bank Group contributes—the Bank Group’s contribution to higher-level outcomes amounts to more than the sum of its parts. And ratings are not the same as outcomes.
IFC, under its 3.0 strategy, has adopted tools to help it more clearly work toward outcomes. Through its AIMM system, IFC assesses all investments for their anticipated direct and indirect effects, including catalytic effects on markets. It is too early to tell how IFC’s new tools will influence outcome achievement and incentives. Although IFC can ensure alignment between projects and IFC’s higher-level goals because of its focused business model, the World Bank operates with objectives that are more diverse because of its diverse sector and country contexts.
Results in thematic areas focus more on compliance than outcomes
Consider gender and climate change. The Bank Group is committed to achieving important outcomes for its global work in key areas such as gender and climate change. Its measurement systems, however, place more emphasis on the input level—such as compliance with climate co-benefit commitments and gender tags—than on capturing changes arising from Bank Group operations.
The climate change results measurement system, for example, uses 35 targets and indicators to monitor how well the WBG integrates climate change into operations and strategies. 90 percent of these indicators relate to actions under the Bank Group’s control, including inputs (such as financing for climate action), internal processes (such as greenhouse gas accounting), and outputs (such as the number of activities that support cities with climate-related policies).
This approach is useful to track fulfillment of corporate commitments, drive accountability, and ensure operations adhere to process requirements. In a nutshell, the system creates incentives to mainstream climate action. But the system has limited focus on the quality of programs and on understanding the higher-level change the programs are meant to support.
What an outcome orientation would mean
Imagine a situation where staff spend less time checking boxes and collecting data that the Bank Group needs for its own internal reporting purposes and have more time engaging with clients on programs’ contribution to development outcomes. Imagine a situation where data, evidence, and systems oriented the Bank Group toward achieving higher-level outcomes. We call this outcome orientation.
An outcome orientation means both generating evidence on what works, what does not, and why; and using this feedback to engage clients and adapt programs to boost contribution to development outcomes.
Many parts of the World Bank Group are already studying their outcomes, learning from data and evidence, and adjusting interventions. Some of the corporate results systems could better support staff in this. IEG’s newly published evaluation on The World Bank Group Outcome Orientation at the Country Level finds that the country-level results measurement system does not effectively support teams in making course corrections to country programs.
***
The World Bank Group’s results measurement systems bring rigor and discipline to performance assessment, and there remains a strong case for assessing Bank Group achievements at the project level so that they can be accounted for in a focused way.
At the same time, more deliberately contributing to development outcomes requires shifting focus beyond the project level and beyond current results measurement and incentives structures. IEG is to this end committed to supporting the World Bank Group’s outcome orientation.
Read Results and Performance of the World Bank Group 2020
The World Bank Group Outcome Orientation at the Country Level
Web Resource
This learning-focused evaluation provides a new vision of how to strengthen the World Bank Group's outcome orientation in countries.
This learning-focused evaluation provides a new vision of how to strengthen the World Bank Group's outcome orientation in countries.