Many World Bank Group projects display strong performance. These results however tell us little about the Bank Group’s aggregate contributions and higher-level development outcomes—the results measurement systems are designed to tell a different type of performance story.

The World Bank Group’s (WBG) results measurement systems for projects, programs, and thematic areas are purposefully designed.

Generally, they focus on collecting data on indicators that can be attributed to Bank Group interventions. They help verify whether targets have been met. They provide consistency in displaying results across portfolios. Importantly, they allow shareholders to hold the Bank Group accountable.

In our flagship report, IEG aggregates and analyzes much of this data to give the big picture on the Results and Performance of the World Bank Group (RAP 2020).

The 2020 RAP report finds that the World Bank and MIGA performs strongly overall, and that IFC’s decline in ratings that started ten years ago appears to have halted. We welcome you to dive deeper into how various arms and instruments of the Bank Group perform, and what factors help explain these trends.

At the same time, while being able to attribute results to Bank Group projects remains important for reasons of accountability, there is also a need to better capture how the institution contributes to lasting positive change in client countries.

We refer to this as higher-level development outcomes—social and economic change on the ground which Bank Group interventions cannot achieve singlehandedly, but which the institution can meaningfully add to, over time.

How well does the World Bank Group contribute to these sorts of development results? RAP 2020 finds that answering this question would require searching beyond the institution’s existing results measurement systems.

Limited evidence on higher-level outcomes is gathered

Following an analysis of the Bank Group’s results reporting—from projects to country programs to thematic priorities such as gender and climate change—the RAP 2020 arrived at the conclusion that the World Bank Group collects limited systematic evidence on its contribution to higher-level outcomes.

Higher-level outcomes stem from the interplay of different projects and types of Bank Group engagements—lending, knowledge, and convening—over time. The World Bank Group’s Board has requested more evidence on these sorts of development outcomes and how the interventions help achieve the Sustainable Development Goals (SDGs). Better evidence on higher level outcomes would also help with learning, reflections on strategy, and course corrections where needed.

Many of the World Bank Group’s existing corporate results measurement systems were designed to collect data needed for ratings and for process and compliance monitoring. These systems do what they were designed to do.

For example, individual projects’ self-evaluations mostly do a good job at capturing project-level results. IEG rates these self-evaluations, as part of a system that provides consistency in displaying results across sectors, across types of projects, and across time. Yet combining performance ratings from individual projects cannot give a clear picture of the important development outcomes to which the Bank Group contributes—the Bank Group’s contribution to higher-level outcomes amounts to more than the sum of its parts. And ratings are not the same as outcomes.

IFC, under its 3.0 strategy, has adopted tools to help it more clearly work toward outcomes. Through its AIMM system, IFC assesses all investments for their anticipated direct and indirect effects, including catalytic effects on markets. It is too early to tell how IFC’s new tools will influence outcome achievement and incentives. Although IFC can ensure alignment between projects and IFC’s higher-level goals because of its focused business model, the World Bank operates with objectives that are more diverse because of its diverse sector and country contexts.

Results in thematic areas focus more on compliance than outcomes

Consider gender and climate change. The Bank Group is committed to achieving important outcomes for its global work in key areas such as gender and climate change. Its measurement systems, however, place more emphasis on the input level—such as compliance with climate co-benefit commitments and gender tags—than on capturing changes arising from Bank Group operations.

The climate change results measurement system, for example, uses 35 targets and indicators to monitor how well the WBG integrates climate change into operations and strategies. 90 percent of these indicators relate to actions under the Bank Group’s control, including inputs (such as financing for climate action), internal processes (such as greenhouse gas accounting), and outputs (such as the number of activities that support cities with climate-related policies).

This approach is useful to track fulfillment of corporate commitments, drive accountability, and ensure operations adhere to process requirements. In a nutshell, the system creates incentives to mainstream climate action. But the system has limited focus on the quality of programs and on understanding the higher-level change the programs are meant to support.

What an outcome orientation would mean

Imagine a situation where staff spend less time checking boxes and collecting data that the Bank Group needs for its own internal reporting purposes and have more time engaging with clients on programs’ contribution to development outcomes. Imagine a situation where data, evidence, and systems oriented the Bank Group toward achieving higher-level outcomes. We call this outcome orientation.

An outcome orientation means both generating evidence on what works, what does not, and why; and using this feedback to engage clients and adapt programs to boost contribution to development outcomes.

Many parts of the World Bank Group are already studying their outcomes, learning from data and evidence, and adjusting interventions. Some of the corporate results systems could better support staff in this. IEG’s newly published evaluation on The World Bank Group Outcome Orientation at the Country Level finds that the country-level results measurement system does not effectively support teams in making course corrections to country programs.

***

The World Bank Group’s results measurement systems bring rigor and discipline to performance assessment, and there remains a strong case for assessing Bank Group achievements at the project level so that they can be accounted for in a focused way.

At the same time, more deliberately contributing to development outcomes requires shifting focus beyond the project level and beyond current results measurement and incentives structures. IEG is to this end committed to supporting the World Bank Group’s outcome orientation.

Read Results and Performance of the World Bank Group 2020