The Results Agenda Needs a Steer—What Could Be its New Course?
Web Resource
Is it time to rethink Results Based Management in international development? We imagine a system that favors learning over compliance.Is it time to rethink Results Based Management in international development? We imagine a system that favors learning over compliance.
What the World Bank Group’s Performance Results Cannot Tell Us About Development Outcomes
Web Resource
Many World Bank Group projects display strong performance. These results however tell us little about the Bank Group’s aggregate contributions and higher-level development outcomes—the results measurement systems are designed to tell a different type of performance story.
The World Bank Group’s (WBG) results measurement systems for projects, programs, and thematic areas are purposefully Show MoreMany World Bank Group projects display strong performance. These results however tell us little about the Bank Group’s aggregate contributions and higher-level development outcomes—the results measurement systems are designed to tell a different type of performance story.
The World Bank Group’s (WBG) results measurement systems for projects, programs, and thematic areas are purposefully designed.
Generally, they focus on collecting data on indicators that can be attributed to Bank Group interventions. They help verify whether targets have been met. They provide consistency in displaying results across portfolios. Importantly, they allow shareholders to hold the Bank Group accountable.
In our flagship report, IEG aggregates and analyzes much of this data to give the big picture on the Results and Performance of the World Bank Group (RAP 2020).
The 2020 RAP report finds that the World Bank and MIGA performs strongly overall, and that IFC’s decline in ratings that started ten years ago appears to have halted. We welcome you to dive deeper into how various arms and instruments of the Bank Group perform, and what factors help explain these trends.
At the same time, while being able to attribute results to Bank Group projects remains important for reasons of accountability, there is also a need to better capture how the institution contributes to lasting positive change in client countries.
We refer to this as higher-level development outcomes—social and economic change on the ground which Bank Group interventions cannot achieve singlehandedly, but which the institution can meaningfully add to, over time.
How well does the World Bank Group contribute to these sorts of development results? RAP 2020 finds that answering this question would require searching beyond the institution’s existing results measurement systems.
Limited evidence on higher-level outcomes is gathered
Following an analysis of the Bank Group’s results reporting—from projects to country programs to thematic priorities such as gender and climate change—the RAP 2020 arrived at the conclusion that the World Bank Group collects limited systematic evidence on its contribution to higher-level outcomes.
Higher-level outcomes stem from the interplay of different projects and types of Bank Group engagements—lending, knowledge, and convening—over time. The World Bank Group’s Board has requested more evidence on these sorts of development outcomes and how the interventions help achieve the Sustainable Development Goals (SDGs). Better evidence on higher level outcomes would also help with learning, reflections on strategy, and course corrections where needed.
Many of the World Bank Group’s existing corporate results measurement systems were designed to collect data needed for ratings and for process and compliance monitoring. These systems do what they were designed to do.
For example, individual projects’ self-evaluations mostly do a good job at capturing project-level results. IEG rates these self-evaluations, as part of a system that provides consistency in displaying results across sectors, across types of projects, and across time. Yet combining performance ratings from individual projects cannot give a clear picture of the important development outcomes to which the Bank Group contributes—the Bank Group’s contribution to higher-level outcomes amounts to more than the sum of its parts. And ratings are not the same as outcomes.
IFC, under its 3.0 strategy, has adopted tools to help it more clearly work toward outcomes. Through its AIMM system, IFC assesses all investments for their anticipated direct and indirect effects, including catalytic effects on markets. It is too early to tell how IFC’s new tools will influence outcome achievement and incentives. Although IFC can ensure alignment between projects and IFC’s higher-level goals because of its focused business model, the World Bank operates with objectives that are more diverse because of its diverse sector and country contexts.
Results in thematic areas focus more on compliance than outcomes
Consider gender and climate change. The Bank Group is committed to achieving important outcomes for its global work in key areas such as gender and climate change. Its measurement systems, however, place more emphasis on the input level—such as compliance with climate co-benefit commitments and gender tags—than on capturing changes arising from Bank Group operations.
The climate change results measurement system, for example, uses 35 targets and indicators to monitor how well the WBG integrates climate change into operations and strategies. 90 percent of these indicators relate to actions under the Bank Group’s control, including inputs (such as financing for climate action), internal processes (such as greenhouse gas accounting), and outputs (such as the number of activities that support cities with climate-related policies).
This approach is useful to track fulfillment of corporate commitments, drive accountability, and ensure operations adhere to process requirements. In a nutshell, the system creates incentives to mainstream climate action. But the system has limited focus on the quality of programs and on understanding the higher-level change the programs are meant to support.
What an outcome orientation would mean
Imagine a situation where staff spend less time checking boxes and collecting data that the Bank Group needs for its own internal reporting purposes and have more time engaging with clients on programs’ contribution to development outcomes. Imagine a situation where data, evidence, and systems oriented the Bank Group toward achieving higher-level outcomes. We call this outcome orientation.
An outcome orientation means both generating evidence on what works, what does not, and why; and using this feedback to engage clients and adapt programs to boost contribution to development outcomes.
Many parts of the World Bank Group are already studying their outcomes, learning from data and evidence, and adjusting interventions. Some of the corporate results systems could better support staff in this. IEG’s newly published evaluation on The World Bank Group Outcome Orientation at the Country Level finds that the country-level results measurement system does not effectively support teams in making course corrections to country programs.
***
The World Bank Group’s results measurement systems bring rigor and discipline to performance assessment, and there remains a strong case for assessing Bank Group achievements at the project level so that they can be accounted for in a focused way.
At the same time, more deliberately contributing to development outcomes requires shifting focus beyond the project level and beyond current results measurement and incentives structures. IEG is to this end committed to supporting the World Bank Group’s outcome orientation.
Read Results and Performance of the World Bank Group 2020
The World Bank Group Outcome Orientation at the Country Level
Web Resource
This learning-focused evaluation provides a new vision of how to strengthen the World Bank Group's outcome orientation in countries.
This learning-focused evaluation provides a new vision of how to strengthen the World Bank Group's outcome orientation in countries.
Results and Performance of the World Bank Group 2020
Web Resource
This report, also known as RAP 2020, is an annual review of the development effectiveness of the World Bank Group. To provide new perspectives on performance, RAP 2020 also analyzes outcomes and discusses ways in which the Bank Group can continue to enhance its outcome orientation.
This report, also known as RAP 2020, is an annual review of the development effectiveness of the World Bank Group. To provide new perspectives on performance, RAP 2020 also analyzes outcomes and discusses ways in which the Bank Group can continue to enhance its outcome orientation.
Convening for Peace: Lessons from Evaluating the World Bank Group
Web Resource
More and more, the World Bank Group is contributing to international collective action to realize Sustainable Development Goal 16 for just and peaceful societies. A recent evaluation assesses the Bank Group’s global engagements of this kind. It finds that the Bank Group is a sought-after global player. Aligning global convening efforts with in-country programs, and monitoring them Show MoreMore and more, the World Bank Group is contributing to international collective action to realize Sustainable Development Goal 16 for just and peaceful societies. A recent evaluation assesses the Bank Group’s global engagements of this kind. It finds that the Bank Group is a sought-after global player. Aligning global convening efforts with in-country programs, and monitoring them systematically, could further benefit the World Bank Group’s convening for peace.
This week the World Bank will wrap up its Fragility Forum, a biennial event that brings together practitioners and policymakers from around the world to exchange knowledge about engaging in contexts affected by fragility, conflict and violence (FCV).
The World Bank Group’s ability to bring together, or convene, actors on major global issues this way is an example of the convening power it holds.
Alongside its capacity to mobilize financing and provide advisory and analytical services to address development challenges, the Bank Group’s role as a global convener is a cornerstone of its value proposition to clients and shareholders. How well does the Bank Group deploy its convening power? IEG recently explored this.
We assessed how the World Bank Group convenes international partners to act collectively on global issues critical to its mission. This is a first-of-its-kind evaluation, that explores what global issues the Bank Group convenes on, what factors drive its convening choices, and what factors determine its convening effectiveness.
We found that that the World Bank Group is increasingly engaging in efforts that relate to fragile contexts, driven by high demands from shareholders and donors to help achieve Sustainable Development Goal 16 (SDG 16) on peace. The Bank Group largely meets these demands, assuming the role of a responsive global convener.
Aligning the World Bank Group’s global and country-level work
Stakeholders typically request the Bank Group to work in tandem with other specialized international organizations, particularly the UN, when convening around FCV issues. Our evaluation found that the Bank Group’s convenings on many such themes – including crisis response, forced displacement, and the humanitarian-development-peace nexus – are indeed based on strong collaborations with different development partners, including the UN.
{"preview_thumbnail":"/sites/default/files/Data/styles/video_embed_wysiwyg_preview/public/video_thumbnails/us-rtIMc4Ro.jpg?itok=uRrLyqDT","video_url":"https://youtu.be/us-rtIMc4Ro","settings":{"responsive":0,"width":"854","height":"480","autoplay":0},"settings_summary":["Embedded Video (854x480)."]}
A recent IEG evaluation finds that the World Bank Group has strong comparative advantages in catalyzing action on global agendas.
Some of the Bank Group’s financial mechanisms to address FCV and forced displacement respond to demand from prominent stakeholders to help shape multilateral responses to these issues. Financial mechanisms such as the State and Peacebuilding Fund, the Global Concessional Financing Facility for middle-income countries, and IDA, including IDA’s Sub-Window for Refugees and Host Communities, help make the Bank a stronger convener on FCV issues.
At the same time, our interviews and case studies identified weaker translation of these global agendas into country-level engagements. While at times this can be due to political sensitivities of operating in FCV contexts, our findings suggest that internally within the World Bank Group, the global work could benefit from more consistent reflection in country programs. This could help ensure better results on the ground.
At times, the Bank Group’s country engagement model can be limiting when addressing challenges that cross national boundaries. World Bank projects predominantly implement country-focused solutions – improving coordination across the Bank’s country teams, and strengthening ownership of regional programs among partner governments, could benefit the global work.
Improving accountability for convening results
The share of the World Bank’s operating budget going to global engagements is around 13 percent. Yet there is no clear system to track convening initiatives and results.
Successful global convening should lead to outcomes such as shared understanding, or changes in positions and attitudes; shared solutions, or negotiated changes in standards, policies, and financing practices; and shared implementation, or setting up programs and partnerships to finance and coordinate given development challenges.
In the absence of tracking systems, managerial attention to the convening portfolio risks being uneven and less systematic. Attention gets paid to some prominent initiatives and many of the formal partnership programs.
However, there is less oversight of convening initiatives when they are managed below the corporate level, at the department or vice-presidential unit levels. This occurs because convening initiatives sometimes lack explicitly stated objectives, success cannot be measured easily, and managing units face relatively weak accountability for their performance.
To improve the effectiveness of global convening, including on efforts to support Sustainable Development Goal 16, corporate processes and systems could better support managing convening initiatives over their life cycle. Many of the global and regional initiatives that the World Bank Group convenes in the space of fragility, conflict, and violence are relatively recent, and some have already passed their piloting phase. It is critical to have these initiatives periodically assessed to ensure better selectivity of global engagements and a focus on results.
Learn more about the effectiveness of the World Bank Group’s global convening in The World’s Bank: An Evaluation of the World Bank Group’s Global Convening. The report seeks to inform discussions about the Bank Group’s role as a major actor on global development policy issues at a time when demand for collective response to crises is increasing but support for multilateralism from major powers is fragile.
To read about the Bank Group’s convening on issues related to FCV, please see Appendix E of the evaluation and the World Bank Group’s FCV Strategy.
Image credit: Andrea Schmitz
The World Bank and global collaboration: Lessons for the COVID-19 (coronavirus) response
Web Resource
The coronavirus pandemic (COVID-19) calls for an international response. But when and why does collective action work best? We studied the World Bank Group’s global collaborations to find out.
The global health ecosystem is complex, with many actors, initiatives and competing priorities. The World Bank has been navigating this landscape and supporting a wide spectrum of collaborations for Show MoreThe coronavirus pandemic (COVID-19) calls for an international response. But when and why does collective action work best? We studied the World Bank Group’s global collaborations to find out.
The global health ecosystem is complex, with many actors, initiatives and competing priorities. The World Bank has been navigating this landscape and supporting a wide spectrum of collaborations for decades.
In the early 2000s, the Bank helped create the Global Fund and GAVI, the Vaccine Alliance. Since then, the Bank has found itself contributing to important global public health issues, including responding to many epidemics that are now household names – SARS, MERS, Ebola, Avian Flu, and others.
Unlike these past epidemics, the COVID-19 pandemic has rapidly morphed into a global health and economic crisis. Addressing the multiple impacts of the pandemic will require collective action on a greater scale and bringing together, or convening, multiple actors to draw on their respective comparative advantages.
Lessons from Evaluating “the World’s bank”
The Independent Evaluation Group recently finished a major evaluation of the World Bank Group’s global convening. Though the report was wrapped up just before the COVID-19 outbreak, some of its key findings and recommendations are relevant to the global response:
We found high demand for the World Bank Group’s global convening. The high demand is because partners see the need to come together to develop joint solutions to pressing challenges and trust the Bank Group to do a good job. Trust is always important in a crisis. Now, more than ever, people look for experts and organizations that they can trust to lead.
Several factors drive effectiveness. Our evaluation found that the Bank Group’s convening is more likely to be effective when global partners share a common understanding and sense of urgency that collective action is needed; internal capacities are strong; and initiatives have clear objectives, links to country programs, and sustained engagement. These conditions are all present for the World Bank Group’s COVID-19 response.
There is value in focus and continuity. A clear sense of the specific goals of the Bank Group’s convening and the scope of its engagements are essential foundations for effective global work. If an organization, any organization, tries to do everything, it does nothing well. If debt relief for the poorest countries is the key goal, stick to that goal for some time. Don’t introduce too many competing goals, and don’t abandon the goal before it is within sight.
Set goals and track progress. The Bank Group often does not give itself enough credit for the results of its global work. When there often are no clear goals for the global work, and no tracking of progress, reporting the results becomes impossible. It would help the World Bank Group to better track, assess, and report the results of its global work.
Manage tensions between – and within – organizations. There is some degree of tension and competition over roles and mandates in the global community. Tension may arise with other organizations as both they and the World Bank Group seek, or are perceived to seek, pieces of the COVID-19 agenda and the organizational prestige that comes from being at the forefront of the crisis response. Also, inside the World Bank, units may compete to stake out their piece of the action. Tensions are not necessarily a bad thing, but they do need to be managed.
An understanding of when collective action works best and why, with the same focus on results applied to global and other work, will help lay the foundations for even more effective joint efforts to address a host of global challenges.
A recent IEG evaluation traces the successes of international collaboration in tackling global challenges.
Laying the groundwork for global teamwork
Digging down into the initiatives that are at the forefront to help developing countries cope with COVID-19, reveals many of the footprints of the World Bank Group’s global work.
These build on years of concerted effort setting up partnerships, cooperation platforms, data exchanges, and financial mechanisms that allow countries and organizations to join forces on shared problems.
Recent examples are the Global Preparedness Monitoring Board, which was co-convened by the Bank and WHO to ensure preparedness for global health crises, and the Coalition for Epidemic Preparedness Innovations (CEPI), a global mechanism to finance and co-ordinate vaccine development for which the World Bank is a trustee. While these were launched in 2018 and 2017 respectively, they are in fact the fruits of the World Bank’s focused and sustained global engagements in health.
The G20’s push for bilateral debt relief for poor countries, to free up resources to cope with the impacts of the coronavirus, is another example of current global convening. This can be traced back to the World Bank Group’s and the International Monetary Fund’s convening of G20 member countries over many years, working on debt relief, crisis responses and diverse initiatives in many sectors.
The global effort to address the COVID-19 pandemic will face many more challenges, and drawing on lessons from past experiences of convening can help individual actors navigate the complex terrain of collective action.
The World Bank plays a large convening role in global health issues. Learn more in Appendix F of The World’s Bank: An Evaluation of the World Bank Group’s Global Convening.
Read the full evaluation
Image Credit: adapted from shutterstock/ GoodStudio and shutterstock/ Marish
The World’s Bank: An Evaluation of the World Bank Group’s Global Convening
PDF file
This first-of-its-kind evaluation assesses the scope and effectiveness of the World Bank Group’s convening on major global development challenges.This first-of-its-kind evaluation assesses the scope and effectiveness of the World Bank Group’s convening on major global development challenges.
Social Contracts and World Bank Country Engagements: Lessons from Emerging Practices
PDF file
The objective of this evaluation is to take stock of social contract knowledge to assess the World Bank’s role in helping countries reshape their social contracts, especially through the integration of social contract diagnostics into country engagements.The objective of this evaluation is to take stock of social contract knowledge to assess the World Bank’s role in helping countries reshape their social contracts, especially through the integration of social contract diagnostics into country engagements.
From what works to how, and for whom
Web Resource
How we can better connect evaluation and evidence to meaningful outcomes.How we can better connect evaluation and evidence to meaningful outcomes.
Conversations: Creating Markets, what are the drivers of success?
Web Resource
An expert panel discussion about the factors underlying the success of IFC's new corporate strategy.An expert panel discussion about the factors underlying the success of IFC's new corporate strategy.