The Results and Performance of the World Bank Group (RAP) report is IEG's annual review of the development effectiveness of the World Bank Group (WBG), which includes IBRD/IDA, IFC, and MIGA. The report synthesizes existing evidence from IEG evaluations, validations, and other products, complemented by relevant information from other sources (e.g., WBG corporate documents).
Beginning with RAP 2013, the first chapter of each year's RAP has focused on a special theme. This year's report focuses on selected questions related to the contribution of the World Bank Group to environmental sustainability.
In this brief interview, IEG Director-General and World Bank Senior Vice President for Evaluation Caroline Heider gives us a brief summary of the 2017 RAP report, and reflects on its implications for the World Bank Group.
This week, IEG published its annual results and performance of the World Bank Group report. Tell us more about this report and why it matters.
The RAP, as we call it, matters because it checks annually the “pulse” of the World Bank Group’s portfolio of projects. At a glance, shareholders, management, and other stakeholders can tell whether the portfolio is meeting its goals, client countries are benefiting from the operations as intended, and the development effectiveness of the Group is as expected. The report also discusses performance to explain the drivers of success and failure with the intention to identify opportunities for course-corrections to achieve better results.
Read the Report
How is this report different than the Bank Group’s own annual report? Is there any connection between the two?
The RAP summarizes data from a portfolio of projects that was approved roughly seven years ago as compared to the current and forward-looking aspects of the Bank Group’s annual report.
At times we are asked whether these “old” projects are still relevant.
My answer is yes.
For one, when investment projects are completed at the World Bank, they start their normal operational life that lasts often 20-30 years. For instance, a road project is “operational” at the World Bank during its construction phase; but its true operational life, namely when it is in use starts after. A completion report that shows poor results does not bode well for the long-term performance, rates of return, and development results of projects.
Secondly, when we look at the portfolio rather than individual projects, we see larger patterns that point to institutional or systemic issues, such as processes or well-ingrained practices. For instance, we have flagged for several years now problems with monitoring systems that undermine achieving development outcomes. But, fixing systemic issues is difficult and time-consuming, so that findings even from so-called “old” projects continue to be relevant.
This year’s report took a close look at the environmental sustainability of World Bank Group’s portfolio. Is the Bank Group’s portfolio getting greener?
It is, but only gradually. In our report, we do not evaluate the percentage increase – 4 percent, comparing period FY08-10 with period FY15-17 – because the World Bank Group does not have a target. But, the report invites the shareholders and management to discuss whether this is green enough or they want to see greater investments in the clean, green, and resilient agendas.
The report looks separately at IFC, World Bank (IBRD and IDA), and MIGA. How does the performance break out by institution?
The World Bank meets its targets when the portfolio is analyzed by dollar amounts committed, but not when we look at the number of projects. This means that larger projects are producing better results than smaller ones. Two years ago, we wanted to know what explains this pattern. Were larger projects better from the outset, maybe got more resources for preparation? We found that these large, successful projects had started out roughly the same size as others, but had received additional resources when they were highly performing.
The IFC portfolio has seen a second year of decline. Over the past 18 months, we spent time with IFC to unpack the reasons for the decline. These analyses identified a number of systemic issues, which have been discussed at various levels to identify necessary actions to improve portfolio performance and results. It will take a couple of years for the trends to turn around, but IFC tackles the problems with the same commitment they have shown as in discussing with us solutions, they should be on a good path.
MIGA’s performance has been steady for the last 10 years at 62 percent.
Are there other variations in the performance trends, for example by region and sector?
Ratings within World Bank regions varied. Comparing ratings within each region (between the group of projects closed in FY11-13 and those closed in FY14-16) indicates improvements in the Africa region and the East Asia and Pacific region, both by number of projects and by volume.
Ratings within each Global Practice also varied over time. By volume, most Global Practices improved on outcome ratings and World Bank performance ratings between the two periods. However, four Global Practices did not improve on outcome ratings and four did not improve on World Bank performance ratings.
What explains these differences?
Since the RAP is a review (relying mainly on existing evaluative information rather than generating new information), the reasons for ratings patterns for specific Regions or Global Practices are not explored in the document. RAP 2017, did, however, use a structured literature review of sources inside the World Bank and in peer-reviewed literature outside the World Bank to answer this question by looked at the degree of consensus about factors associated with development performance of World Bank projects. Commonly identified project-level success factors included supervision quality and the ratings record of projects managed by the task team leader.
Other success factors were incorporation of lessons learned and analytical work, quality of the project’s results chain, and quality of project design. For community-driven development, the extent and quality of collaboration with local stakeholders was identified as essential to project success.
Factors associated with less successful projects included projects with early warning flags, or projects of longer durations. Also, project outcome was negatively associated with the cost and time of both preparation and supervision. At the country level, the most commonly identified success factor was economic growth, while project success is less likely in fragile and conflict-affected situations and when civil liberties are limited.
In your view, where should the institution be investing its efforts to improve development outcomes?
Based on our analyses, outcome ratings are highly dependent on what we call “quality at entry”, meaning the quality of project design when it is approved. This finding is not surprising. Put simply, it means that the better the preparations are, the better the results will be.
What do we mean by that?
Strong diagnostics of the development challenge at hand, a clear articulation of intended results that is shared by stakeholders in the country and the World Bank Group, and metrics that help measure and monitor whether things are on track simply help align efforts towards goals. That’s key to success, and it also helps manage in uncertain times when tactics need to adjust to changing circumstances. It is like sailing. Once you set your course towards a goal, wind, currents, and waves might force you to change your tactics to get there. But, you won’t get anywhere if skipper and crew have no or different ideas about where they are heading; then, at best progress will be slow, at worst the boat will break apart.
Does IEG have a role in ensuring that the Bank Group’s management acts on your report? What’s the mechanism for translating the report’s findings into potential corrective action?
The RAP does not provide recommendations. The reason is that is provides a synthesis of other evaluations – trend data comes from project reporting, theme chapters are based on sector, thematic, and corporate evaluations – which have made recommendations. Nonetheless, as mentioned previously, in the case of IFC, we had an intense set of engagements to discuss how IFC can turn around its portfolio. A promising start; we will see how their efforts are reflected in results and performance in a couple of years’ time.
In addition, we hope the discussion of how “green” the portfolio is will stimulate discussions among shareholders and management to decide whether they want to raise the bar.
More importantly, though, is this year’s third chapter, which we have not yet talked about. It provides the usual summary of follow-up actions the World Bank Group has taken in response to our recommendations. But, this year we have piloted something new. In line with the theme chapter, we did a retrospective of 10 years of recommendations on environmental sustainability. We wanted to understand what kind of recommendations we had made over time, and whether things have changed at the World Bank Group since then. The review has obvious limitations – we have not corroborated our analyses with direct observations in the field – but it shows interesting patterns that are useful to us and to managements. The analysis led to useful discussions as yet another step to support learning from independent evaluation.