Highlights

This Results and Performance of the World Bank Group updates the annual analysis of project portfolio performance using the Independent Evaluation Group’s database of project evaluations (Implementation Completion and Results Report Reviews and Project Performance Assessment Reports).

It focuses on the Bank Group’s effectiveness at the country program level and the use or influence of advisory services and analytics.

The new country engagement guidance adopted by the Bank Group in 2021 did not yet apply to the country programs analyzed in this report. Instead, this Results and Performance of the World Bank Group establishes a baseline against which country program performance resulting from the new guidance can be measured in the future.

The primary data sources for the country-level analysis are the Independent Evaluation Group’s Completion and Learning Review (CLR) Reviews and Country Program Evaluations.

Context

This year’s Results and Performance of the World Bank Group (RAP) report updates the project portfolio performance undertaken in previous RAPs (World Bank 2020a, 2021b) while focusing on the country level. It presents an annual review of evidence from Independent Evaluation Group (IEG) evaluations and validation work on the development effectiveness of the Bank Group.1 The 2022 RAP is the 12th in an annual series.

The 2022 RAP aligns with the Bank Group’s outcome orientation agenda by focusing on the country level.2 This country-level focus also responds to the interest of members of the Bank Group’s Board of Executive Directors for reporting specifically on country-level performance. Accordingly, this RAP presents a snapshot of the Bank Group’s effectiveness at the country program level, covering lending and advisory services and analytics (ASA). A 10-year period (from fiscal year [FY]13 to FY22) was chosen because this is the first time such country-level analysis is being undertaken. The revised country engagement guidance that the Bank Group adopted in July 2021 did not yet apply to the country programs analyzed in this RAP. Instead, this RAP establishes a baseline against which country program performance resulting from the new guidance can be measured in the future.

This RAP also focuses on the performance of the Bank Group with regard to the World Bank Group performance rating (for the country level) and the Bank performance rating (for the project level) as well as a qualitative analysis of performance, and it aims to bring attention to this dimension of development effectiveness. This is because the performance of the Bank Group is fully within the Bank Group’s control and can yield direct insights into what the Bank Group can do to enhance the future effectiveness of its lending and ASA support.

Methods and Data Sources

The primary data sources for this year’s RAP are IEG’s Completion and Learning Review (CLR) Reviews and Country Program Evaluations (CPEs). All CLR Reviews were examined for the ratings trends analysis. A stratified random sample of 50 countries was chosen for in-depth qualitative analysis from a population of 108 countries with 152 CLR Reviews completed during FY13–22 (with a cut-off date of March 7, 2022).3 IEG reviewed the most recent CLR Reviews for each of these 50 countries (appendix A). All 9 completed CPEs were also examined for the same analysis.4 Furthermore, the findings from CPEs and CLR Reviews were complemented by country-level insights from a purposive sample of sector and thematic evaluations (including corporate evaluations), Project Performance Assessment Reports, and other relevant evaluative material. For the International Finance Corporation (IFC) and the Multilateral Investment Guarantee Agency (MIGA), the findings from CPEs and CLR Reviews were complemented by country-level insights from sector highlights, learning engagements, and synthesis notes. Appendix B provides a count of available IEG evaluations by year.

An important caveat relates to the RAP’s use of IEG’s CLR Reviews, which are based on the Bank Group’s Completion and Learning Reviews (CLRs). IEG’s evaluation of outcome orientation at the country level pointed out that the Bank Group’s CLRs, which IEG’s CLR Reviews validate and rate, provide a partial picture of the Bank Group’s development outcomes at the country level because of their “overemphasis on those results that can be [quantitatively] measured and on results from lending projects” (World Bank 2020b, xiii). Furthermore, the “CLR rarely captures complementarities across instruments or institutions and so is not able to establish whether the Bank Group’s contribution to country outcomes amounts to more than the sum of its parts” (World Bank 2020b, xiii).

At the project level, the project ratings trends were updated from the previous RAP based on IEG’s project ratings database. For the World Bank, the main data sources at the project level were Implementation Completion and Results Report Reviews and Project Performance Assessment Reports. For IFC and MIGA, the main data sources at the project level were IFC Expanded Project Supervision Report Evaluation Notes and IEG Project Evaluation Summaries for IFC investments, IFC Project Completion Report Evaluation Notes for IFC advisory services, MIGA Project Evaluation Report (PER) Validation Notes, and IEG PERs.

For the World Bank, the RAP identified some drivers of performance based on common characteristics of projects using unsupervised machine learning. This analysis was exploratory, and the results are provided in appendix C.

For IFC investments and advisory services projects and MIGA guarantee projects, the RAP identified some drivers of performance at the two ends of the ratings spectrum (for example, highly successful and highly unsuccessful for IFC investments and advisory services projects and excellent and unsatisfactory for MIGA projects).5 Past RAPs have conducted qualitative and manual text analysis, which was complemented by quantitative analyses (regression) analyses. The focus on the two ends of the ratings spectrum in this RAP aimed to elicit any new insights on what worked well and not so well. Because of the small number of evaluated IFC and MIGA projects with development outcome ratings at the two ends of the ratings spectrum, the RAP reviewed the project documents manually. The small number of projects rated at the two ends of the rating spectrum makes it challenging to draw firm conclusions and limits generalizability.

Main Evaluation Questions and Report Structure

The main evaluation questions answered by this RAP and the corresponding chapters are as follows:

• At the project level for the World Bank, how have IEG project ratings (outcome ratings and Bank performance ratings) changed over time and across types of countries and operations? (See chapter 2.)
• At the project level for IFC and MIGA, what explains variations in the ratings at the two ends of the rating spectrum (highly successful and highly unsuccessful for IFC and excellent and unsatisfactory for MIGA)? For IFC investment and advisory services and MIGA guarantee projects (in addition to the previous question), what was the relationship between development outcome and work quality and between development outcome and IFC’s additionality and MIGA guarantees role and contribution? For IFC investments, what was the relationship between development outcome and IFC’s investment outcome? For IFC, what was the relationship between IFC advisory services and investments (particularly those IFC investment projects that had prior or concurrent advisory services)? (See chapter 3.)
• At the country level for the Bank Group, how have IEG country program ratings trends and patterns evolved over time and across types of countries? To what extent did the timing, type, and performance of Bank Group interventions have an impact on the achievement of Country Partnership Framework (CPF) objectives? (See chapter 4.)
• At the country level for the Bank Group, to what extent did the Bank Group’s support (that is, project portfolio and ASA) contribute to the achievement of the CPF’s objectives and the development outcome? To what extent was there a line of sight between the development outcome and high-level outcomes? (See chapter 5.)
• For ASA for the World Bank, how was the use or influence of ASA reported at the country program level? (See chapter 6.)