This year’s Results and Performance of the World Bank Group (RAP), like previous RAPs by the Independent Evaluation Group (IEG), updates the analysis of project portfolio performance. As a new contribution, this 12th edition of the RAP analyzes outcomes and performance at the country program level. The country is the primary unit of engagement of the Bank Group, and this RAP presents the effectiveness of the Bank Group’s support at the country program level. It covers World Bank lending and advisory services and analytics (ASA) and International Finance Corporation (IFC) investment and advisory services and Multilateral Investment Guarantee Agency (MIGA) guarantee operations over a 10-year period (for World Bank, fiscal year [FY]13–22; for IFC advisory services and MIGA guarantee projects, FY12–22; for IFC investment services, calendar years 2012–22).

The Bank Group revised its country engagement guidance in July 2021 to address weaknesses in its approach to country programs, including in the results frameworks of its Country Partnership Frameworks (CPFs). This revised guidance did not yet apply to the country programs analyzed in this RAP. Instead, this RAP establishes a baseline against which the country program performance resulting from the new guidance can be assessed in the future. Building on RAP 2020 and RAP 2021, this RAP updates the project ratings trends since then and carries forward the analysis of outcome orientation at the country level and of the line of sight from Bank Group support to high-level outcomes (HLOs; defined by the Bank Group as a sustained improvement in the well-being of the poorest and most vulnerable people).

Project Portfolio Performance

The analysis of the project portfolio performance draws on IEG’s project databases of Implementation Completion and Results Report Reviews and Project Performance Assessment Reports (for World Bank), IEG’s Evaluation Notes of IFC’s Expanded Project Supervision Reports and Project Completion Reports and IEG’s Project Evaluation Summaries (for the International Finance Corporation), and IEG’s Validation Notes of MIGA Project Evaluation Reports and IEG’s Project Evaluation Reports (for the Multilateral Investment Guarantee Agency).

World Bank. At the time of RAP 2021, 88 percent of projects that closed in FY20 had an outcome rating of moderately satisfactory or higher. However, additional projects that closed in FY20 have been evaluated since the publication of RAP 2021, resulting in a downward revision of the share to 84 percent. For FY21, the percentage of projects rated moderately satisfactory or higher rose marginally, to 85 percent. Among projects rated moderately satisfactory or higher, the proportion of projects rated satisfactory or highly satisfactory increased from 54 percent in FY20 to 58 percent in FY21 (implying a decline in the share of projects rated moderately satisfactory). So far, no decline is observed in the outcome ratings for projects that closed during the COVID-19 pandemic (during FY20 and FY21). For fragile and conflict-affected situations (FCS), the percentage of projects rated moderately satisfactory or higher in FY21 was 74 percent, compared with 82 percent in FY20.

The Bank performance rating and its constituent ratings for quality at entry and quality of supervision have maintained their upward trend. Between FY20 and FY21, Bank performance continued to be strong—the percentage of projects rated moderately satisfactory or higher increased from 86 to 91 percent (80 percent to 85 percent for quality at entry and 86 percent to 92 percent for quality of supervision). Among the projects rated moderately satisfactory or higher, the proportion of projects rated satisfactory or highly satisfactory increased from 41 percent to 45 percent. Furthermore, monitoring and evaluation (M&E) quality—an important dimension of Bank performance—registered clear improvement. Although 57 percent of projects in FY20 were rated high or substantial on M&E quality, 64 percent were rated high or substantial in FY21. M&E ratings for projects in FCS had a similar increase, from 52 percent in FY20 to 57 percent in FY21. However, the Bank performance rating in FCS in FY21 was much lower than the Bank performance rating for all countries in that fiscal year (81 percent moderately satisfactory or higher in FCS, compared with 91 percent for all countries) and, comparing FY21 with FY20, the percentage of projects rated moderately satisfactory or higher declined marginally, from 82 percent to 81 percent.

In addition to reporting on development outcomes, this year’s RAP presents for the first time IFC and MIGA performance on project dimensions such as work quality, role and contribution, or additionality, and for IFC’s investment projects, investment outcome ratings at the project level. The purpose is to assess the performance of IFC and MIGA projects in meeting other institutional objectives, the relationships among these dimensions, and the development outcomes of IFC and MIGA projects.

International Finance Corporation investments. Development outcome ratings of recently evaluated IFC investment projects have improved. Fifty-two percent of investment projects evaluated between calendar years 2019 and 2021 were rated on average mostly successful or higher, compared with 48 percent between 2018 and 2020.

Using IFC’s new regional classification, projects in Central Asia and Türkiye and in South Asia outperformed projects in other regions, especially projects in Africa, whose development outcome success ratings fell from 57 percent in 2012–14 to 31 percent in 2019–21. The low development outcome rating of projects in Africa constrains IFC’s overall development outcome results in International Development Association (IDA) countries. Projects in non-IDA Africa countries have better development outcome ratings. Development outcome ratings of projects in FCS also weakened between 2019 and 2021. Low development outcome ratings of projects in FCS can be observed across all industries, by size of investment commitment and type of IFC investment instrument, and across the four indicators of development outcome. IFC’s additionality ratings in FCS have also declined. IDA countries and FCS are both IFC corporate priorities.

Overall, IFC’s investment outcome success ratings have remained high compared with the ratings on development outcomes, suggesting that even projects with less successful development outcomes ratings still met IFC’s investment returns benchmarks. However, when investment projects perform poorly in delivering development outcomes, IFC’s bottom line also suffers. IFC’s role and contribution or additionality ratings remain stable and closely associated with development outcome ratings. IFC considers additionality essential to achieving development impact, whether in the form of delivering positive outcomes to stakeholders or helping create or develop markets. In 2019–21, the percentage of projects rated satisfactory or higher for IFC’s additionality reached 61 percent. Most of the projects with high additionality ratings benefited from a combination of financial and nonfinancial additionality. IFC’s financial additionality is delivered by way of financial structuring of the investment, while nonfinancial additionality took the form of improved business environment, corporate governance, and environmental and social standards.

IFC investment projects delivered along with IFC advisory services, either sequentially or simultaneously, have better development outcome ratings than stand-alone IFC investment projects. Fifty-five percent of combined investment and advisory services projects have better development outcome ratings, compared with 49 percent for development outcome ratings of IFC investment projects only. However, investment outcome ratings in projects with combined IFC investment and advisory services support are lower, implying that offering these two types of support in a project needs careful consideration and greater selectivity, particularly for advisory services projects that are paired with only equity investment from IFC.

International Finance Corporation advisory services. Development effectiveness ratings of evaluated IFC advisory services projects have steadily improved and stabilized after dropping to their lowest level in 2015–17. Sixty-two percent of projects evaluated in 2019–21 were rated mostly successful or higher, compared with 38 percent in 2015–17.

Performance of IFC advisory services projects in IDA countries was better than projects in non-IDA countries, and development effectiveness ratings of projects in FCS continue to improve, although they remain lower than ratings for projects in non-FCS. As with IFC investment projects, IFC work quality matters in improving the development effectiveness of advisory services projects. Some of the changes made in the advisory services business line after the joint IEG and IFC work quality study in 2017 seem to be making a difference, especially in stronger governance and increased emphasis on lesson learning. Work quality and development effectiveness ratings are closely associated. Forty-three percent of projects rated satisfactory or higher on IFC work quality are also rated highly on development effectiveness. No advisory services projects were rated highly successful on development effectiveness if IFC work quality was poor, and vice versa.

Projects with high ratings on IFC role and contribution also have high ratings on development effectiveness, and the two indicators are closely associated. More than half of advisory services projects rated satisfactory or higher on IFC role and contribution were also rated highly on development effectiveness. The opposite is also true—if IFC role and contribution is rated low, the chance of getting a high development effectiveness rating is slim (only 1 percent likelihood).

Multilateral Investment Guarantee Agency. Development outcome ratings of MIGA projects have improved further, with the share rated satisfactory or better averaging 70 percent of evaluated projects during the six-year period of 2016–21. With the addition of two self-evaluated and validated projects in 2021, the development outcome success rate improved substantially from 66 percent in 2015–20, and it is now one percentage point higher than MIGA’s high success rate in 2013–18. Development outcome ratings of projects in FCS remained the same, but ratings of projects in IDA countries were slightly lower than projects in FCS, non-IDA countries, and non-FCS. MIGA’s work quality ratings are also relatively stable in the six-year periods 2015–20 and 2016–21, with 57 percent and 55 percent of projects rated satisfactory or better, respectively. Work quality also matters in improving MIGA’s development results. Forty-six percent of projects rated highly in MIGA’s work quality also have high development outcome ratings. What MIGA brings to the investment also matters in ensuring positive development outcomes. MIGA’s role and contribution rating reached an all-time high of 88 percent of the 66 self-evaluated and validated projects in 2016–21 rated satisfactory or better (with the caveat that the 2021 MIGA program is ongoing, with only 2 out of 8 projects rated so far). MIGA’s key contributions were in political risk mitigation, enabling provision of long-term financing (and, in some cases, facilitating local currency financing when not available from domestic or international financing sources), mobilizing finance, and increasing foreign direct investment flows. MIGA has also contributed positively to improving the environmental and social aspects of 9 out of 10 recently evaluated projects. The likelihood of getting a high development outcome rating is 66 percent when the rating for MIGA’s role and contribution is also high. Both work quality and role and contribution are aspects within MIGA’s control.

Effectiveness of World Bank Group Support at the Country Program Level

The primary data sources for the analysis at the country program level are IEG’s 152 Completion and Learning Review (CLR) Reviews and 9 Country Program Evaluations completed during FY13–22 (with a cut-off date of March 7, 2022). A stratified random sample of 50 countries—representative of the 108 countries covered by the FY13–22 CLR Reviews—was chosen for in-depth qualitative analysis. There is an important caveat to this analysis. IEG’s evaluation of outcome orientation at the country level pointed out that the Bank Group’s CLRs, which IEG’s CLR Reviews validate and rate, provide a partial picture of country-level development outcomes because of their “overemphasis on those results that can be [quantitatively] measured and on results from lending projects” (World Bank 2020b, xiii). Furthermore, the “CLR rarely captures complementarities across instruments or institutions and so is not able to establish whether the Bank Group’s contribution to country outcomes amounts to more than the sum of its parts” (World Bank 2020b, xiii).

Development outcome ratings at the country program level paint a favorable picture of the Bank Group’s effectiveness. Development outcome ratings have continued to improve since FY14. Among CLR Reviews covered in FY19, FY20, and FY21, all country programs were rated moderately satisfactory or higher, with the caveat that these ratings are drawn from a small number of CLR Reviews (21 CLR Reviews that contained ratings). Regarding COVID-19, the analysis finds that the prepandemic CLR Review ratings are a good predictor of project performance during the COVID-19 period. For FCS, the trend is not reported in this RAP given the very small number of CLR Reviews available for each year.

World Bank Group performance ratings have also continued to improve, though more gradually, and they remain below the development outcome rating. Seventeen of the 21 CLR Reviews, which contained ratings and were covered in FY19, FY20, and FY21, rated World Bank Group performance as good or higher. As noted in the previous paragraph, the FCS trend is not reported in this RAP, given the very small number of CLR Reviews available for each year.

IFC and MIGA contributed 23 percent of the 780 objectives in the 65 CLR Reviews assessed. The share of objectives with IFC and MIGA contribution in recent CLR Reviews increased (from 19 percent to 31 percent) because more attention was paid to the integration of IFC and MIGA activities in the CPFs. Enabling a better business environment and access to finance accounted for half of the CPF objectives with IFC and MIGA contributions. Energy and infrastructure accounted for another quarter of the objectives. IFC uses both investments and advisory services to support the objectives, with business environment objectives supported mainly by advisory services. However, IFC activities that contributed to the CPF objectives represented only a portion of IFC’s active portfolio in a country. In all but one of the objectives, MIGA’s contribution was accompanied by IFC investment or advisory services. MIGA’s business development agreement with IFC emphasizes leveraging IFC’s more extensive global footprint, ensuring collaboration. The objectives with combined IFC and MIGA contributions in energy, finance, and agriculture performed well: 61 percent to 72 percent of rated objectives were assessed as achieved or mostly achieved, compared with 37 percent for business environment. The combined IFC and MIGA contributions for all CPF objectives was assessed as 75 percent achieved or mostly achieved, compared with the 55 percent achieved or mostly achieved rating for all CPF objectives with only IFC or MIGA contributions.

The overall favorable trend in the Bank Group’s effectiveness at the country program level is tempered by the following findings:

• When individual CPF objectives were considered over the 10-year period (FY13–22), nearly half were rated partially achieved or not achieved (based on 113 CLR Reviews for which this information was available).
• There were issues with country program relevance. These included lack of selectivity (such as too many CPF objectives or CPF objectives that were too broad); adaptiveness (such as insufficient preparedness to respond to changes in country conditions, government commitment, or Bank Group priorities); and realism in programs and projects (such as operations that overestimated implementation capacity and underestimated political economy challenges). Furthermore, the CPFs and their results frameworks relied overwhelmingly on the World Bank lending portfolio, and they insufficiently integrated and leveraged ASA and the support provided by IFC and MIGA.
• The One Bank Group approach, wherein support from the World Bank, IFC, and MIGA complement each other, remained a work in progress. There has been a lack of attention in CLRs and CLR Reviews to a discussion of the implementation of the Bank Group’s Mobilizing Finance for Development agenda (formerly Maximizing Finance for Development) and the Cascade approach. With greater emphasis by both IFC and MIGA on the need for collaboration in their corporate strategies, the lack of substantive monitoring of the Mobilizing Finance for Development agenda and the Cascade approach is a missed opportunity for learning.
• A historical lack of ASA monitoring raises questions about the Bank Group’s ambitions to be a “knowledge bank” and its ability to strategically use ASA to improve country-level impact. Analysis of the latest CLR Reviews for a random sample of 50 countries indicated that these CLR Reviews reported, on average, on only about one-third of the ASA program in terms of use or influence.
• The CPFs sometimes did not adapt sufficiently and quickly enough to changes in context, such as when government commitment or Bank Group priorities changed, implementation capacity was weaker than expected, or planned lending failed to materialize, including because of dropped or canceled projects. Even when country programs were adapted to reflect changing country circumstances during CPF implementation, the corresponding Performance and Learning Reviews or results frameworks were not necessarily modified enough to fully reflect the changes. However, the reasons for adaptation are important, and potential trade-offs between adaptation and shifting goalposts need to be recognized and managed.
• The Bank Group was good at identifying risks, especially macroeconomic risks and risks associated with external shocks. It did less well at identifying implementation capacity and political economy risks. Overall, the Bank Group fared less well on risk mitigation than on risk identification. For residual risks and risks that could not be fully mitigated (such as political upheaval risks), the Bank Group lacked procedures or guidance to anticipate possible risk scenarios and propose appropriate program adjustments when those scenarios materialized. Consequently, the Bank Group’s response to residual risk was ad hoc and not adequately informed by how the Bank Group had responded across countries facing similar circumstances.

Line of Sight

The Bank Group defines the line of sight as “a clear path connecting an activity with its ultimate desired outcome” (World Bank Group 2021c, iv). Within the Bank Group context, the line of sight involves two stages: (i) from Bank Group support to CPF objectives/development outcome (the extent of achievement of CPF objectives determines the development outcome rating), and (ii) from CPF objectives/development outcome to HLOs. The line of sight requires two conditions to be met at both stages: relevance (a necessary condition) and contribution (a sufficient condition).

In the first stage, the RAP found that Bank Group support was not always fully relevant to CPF objectives/development outcome due to a lack of selectivity, adaptiveness, or realism. Contribution also had some weaknesses in this first stage in that lending often fell short (that is, planned lending did not materialize or was dropped or canceled, or disbursements were delayed), and adequate restructuring of the country program and the associated results framework was not undertaken at the Performance and Learning Review stage. Stated CPF objectives/development outcome then tended not to be achieved.

The RAP found that both relevance and contribution were less evident in the second stage, even though CPF objectives/development outcome generally mapped well to HLOs. Establishing relevance and contribution in the second stage tends to be difficult because extraneous influences increase as one moves toward outcomes at the final stages, and confounding factors from actions originating outside the Bank Group program come into play. Table O.1 summarizes the RAP’s analysis of the line of sight.

Source: Independent Evaluation Group.

Note: ASA = advisory services and analytics; CPF = Country Partnership Framework; IFC = International Finance Corporation; HLO = high-level outcome; MIGA = Multilateral Investment Guarantee Agency; PLR = Performance and Learning Review.

Paying More Attention to the Performance of the World Bank Group

This RAP supports an increased focus on the Bank Group’s performance at both the project and country levels. This is especially important for the World Bank’s project portfolio in FCS, where the Bank performance rating (81 percent in FY21) trails the Bank performance rating for the entire portfolio (91 percent in FY21). World Bank Group performance, unlike development outcome, is directly within the Bank Group’s control; therefore, a consistently higher rating of Bank performance (for World Bank projects) and World Bank Group performance (at the country level) therefore would not be an unreasonable target. This RAP highlights several factors that can enhance the Bank Group’s performance, including greater selectivity, adaptiveness, and realism in country programs and CPFs and their results frameworks that integrate and leverage ASA and IFC and MIGA support.

World Bank Advisory Services and Analytics

This RAP undertook a qualitative analysis of the use or influence of ASA based on the CLR Reviews for the random sample of 50 countries. More than 80 percent of cases exhibited a good match between ASA topics and the topics covered by government policies and programs, and between ASA topics and topics of CPF objectives/development outcome and World Bank operations. However, when reviewing the CLR Reviews for higher levels of ASA influence (such as ASA influence on policy dialogue, uptake in government programs and policies, and uptake in CPF objectives and World Bank operations), less than half of the CLR Reviews reported on these higher levels of ASA influence. Of those that did, ASA were found to be influential in most cases.

Furthermore, this RAP finds a significant lack of reporting on client and stakeholder ownership and engagement and on dissemination. There were also a few instances of less positive client and stakeholder ownership and engagement. These process issues are important to ensure that the knowledge generated by ASA actually feeds into decision-making. Without such uptake of ASA in decision-making, the World Bank risks being a “report bank,” not yet a “knowledge bank.” Becoming a “knowledge bank” calls for the use of knowledge, which stakeholder engagement and dissemination can foster. Beyond this, knowledge translation or knowledge brokering can help country clients determine which knowledge is most relevant to the decisions they face and how best to tailor global knowledge to local circumstances (White 2019).

Future Directions

Although the enhancements now included in the International Bank for Reconstruction and Development, IFC, MIGA, and IDA country engagement guidance are expected to help address some of the issues identified in this RAP (World Bank Group 2021a), we offer three future directions for the Bank Group’s consideration.

Enhancing the Effectiveness of World Bank Group Support at the Country Program Level

• Improve the selectivity and framing of CPF objectives and the realism of Bank Group country programs, especially regarding implementation capacity and political challenges.
• Prioritize implementation of the One Bank Group approach (including the Mobilizing Finance for Development agenda and the Cascade approach), a long-standing aim of the Bank Group.
• Ensure that the Bank Group adapts and keeps pace with changing circumstances, such as shifts in government commitment or changes in Bank Group priorities, while being mindful not to lower the objectives to make up for the lack of progress. One option would be to consider the application, at the country level, of the M&E system currently applied to restructured projects at the World Bank.
• Report in a timely way, in relevant documents (particularly in the Performance and Learning Review), any adaptations made to the country program and update the results frameworks accordingly.
• Ensure that the Bank Group’s country program benefits from all elements of Bank Group support collectively and that the sum of the parts of Bank Group support is monitored.

Strengthening Risk Identification and Mitigation

• Maintain the Bank Group’s current good performance on identifying macroeconomic risk and risks associated with external shocks while improving the identification of risks associated with implementation capacity and political economy.
• Identify up front the possible key risk scenarios and outline the course of action to address those scenarios so that timely action can be taken, depending on which scenario unfolds.
• Expand and update the current country engagement guidance to include possible key risk scenarios and the responses to each to facilitate better risk mitigation.

Monitoring Advisory Services and Analytics Use or Influence

• Systematically monitor and evaluate whether ASA are achieving their intended influence.
• Consider introducing self-evaluation of analytical work across all Bank Group institutions.