Back to cover

Results and Performance of the World Bank Group 2023

Chapter 1 | Introduction

This Results and Performance of the World Bank Group examines the evolution of the World Bank Group’s project-level results and performance, analyzing the development outcomes and factors that have shaped project implementation and performance, especially in the context of the COVID-19 pandemic.

The methodological approach of this report is underpinned by three principles: continuity, innovation, and symmetry. The report embraces continuity by building on previous work, innovation by incorporating innovative data and evaluation methods, and symmetry by applying them in a balanced and consistent manner across different Bank Group institutions, while taking into account each institution’s different evaluation and rating methods.

This is the 13th annual Results and Performance of the World Bank Group (RAP) report. The RAP comprehensively reviews the development effectiveness of the World Bank Group through evidence gathered by the Independent Evaluation Group (IEG). Specifically, this report assesses the results and performance of the World Bank, which includes the International Bank for Reconstruction and Development and the International Development Association (IDA); the International Finance Corporation (IFC); and the Multilateral Investment Guarantee Agency (MIGA).

RAP 2023 focuses on three main evaluation questions: (i) How did IEG ratings change over time at the project and country levels across the various Bank Group institutions? (ii) What has been the evolution of development outcomes pursued, measured, and achieved at the project level, and what is the relationship of outcomes to project performance ratings? (iii) What factors affected the Bank Group projects’ implementation and performance in the COVID-19 pandemic context? These evaluation questions aimed to respond to specific areas of inquiry that were highlighted by Bank Group management and the Board of Executive Directors during RAP consultations.

This RAP updated project performance rating trends from the past 10 years—fiscal years (FY)12–22—to answer the first evaluation question. These updated performance ratings covered World Bank lending projects, IFC investment and advisory projects, and MIGA guarantee projects (see chapters 2, 3, and 4, respectively).1 Although this is the first RAP with a significant number of projects that took place during the pandemic, many projects were already at an advanced stage of implementation when the pandemic began, minimizing the severity of their exposure. Therefore, this RAP does not fully capture the impact of COVID-19 on projects’ performance, and it is likely that project cohorts in the coming years will reflect the full consequences of the COVID-19 pandemic.

Results and performance across Bank Group institutions are not strictly comparable because of different project evaluation and rating methodologies. For instance, the World Bank projects employ an objective-based methodology to derive project performance ratings. These ratings summarize the World Bank’s self-evaluation and IEG's validation narratives into categories or values that enable aggregation across operations.2 Similarly, IFC’s advisory project performance ratings are derived from an objective-based methodology, which establishes minimum thresholds for rating and assessing these projects’ effectiveness. By contrast, evaluation systems and performance ratings for IFC's investment projects and MIGA's guarantee projects largely rely on a benchmark-based methodology. This benchmark-based methodology aligns with good practice standards for evaluating private sector projects, as established by the Evaluation Cooperation Group of multilateral development banks (ECG 2011). See box 1.1 for a description of the main performance ratings for each institution and appendix A for more details on each institution’s ratings and evaluation methodology.

Box 1.1. Main Performance Ratings in World Bank Investment Projects, IFC Investment and Advisory Projects, and MIGA Guarantee Projects

World Bank

outcome: The extent to which a project efficiently achieved, or was expected to achieve, its relevant objectives. The outcome rating brings together three underlying dimensions: relevance, efficacy (objectives achievement), and efficiency. It is rated on a six-point scale: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory.

relevance: The extent to which a project’s objectives are consistent with current World Bank Group country strategies at the time of project closing. It is rated on a four-point scale: high, substantial, modest, and negligible.

efficacy: The extent to which a project achieves, or was expected to achieve, its objectives, taking into account the objective’s relative importance. The project’s achievement of each individual objective is assessed based on the concept of “plausible causality.” Efficacy ratings also reflect an assessment of the results framework’s validity and use complementary data and evidence on the achievement of intended results. Both the efficacy of each individual objective and overall efficacy in achieving the project development objective are rated on a four-point scale: high, substantial, modest, and negligible.

efficiency: How economic resources and inputs are converted to results. Efficiency ratings indicate whether the costs involved in achieving project objectives were reasonable compared with project benefits and recognized norms (value for money). It is rated on a four-point scale: high, substantial, modest, and negligible.

Bank performance: The extent to which World Bank services ensured quality project design and supported effective implementation through appropriate supervision in the achievement of development outcomes. Bank performance and its two constituent elements—quality at entry and quality of supervision—are rated on a six-point scale: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory.

monitoring and evaluation (M&E) quality: The quality of the design and implementation of project M&E and the extent to which M&E results are used to improve performance. M&E quality is assessed at the project level and includes M&E design, implementation, and use. It is rated on a four-point scale: high, substantial, modest, and negligible.

Figure B1.1.1. Performance Ratings in World Bank Investment Projects

Image

Figure B1.1.1. Performance Ratings in World Bank Investment Projects

Source: Independent Evaluation Group.

Note: This is the ratings structure for investment project financing and Program-for-Results; development policy financing has a slightly modified ratings structure (see appendix A).

IFC

Investment Projects

development outcome: Synthesizes a project’s performance across four subdimensions considering its impact on each affected stakeholder group: project business performance, economic sustainability, environmental and social effects, and private sector development. It is rated on a six-point scale: highly successful, successful, mostly successful, mostly unsuccessful, unsuccessful, and highly unsuccessful.

IFC additionality: The benefit or value addition IFC brings to a project that a client would not otherwise have. It is rated on a four-point scale: excellent, satisfactory, partly unsatisfactory, and unsatisfactory.

IFC investment outcome: The extent to which IFC has realized at the time of evaluation and expects to realize over the remaining life of the investment, the project’s loan income, equity returns, or both based on what was expected at approval. It is rated on a four-point scale: excellent, satisfactory, partly unsatisfactory, and unsatisfactory.

IFC work quality: IFC’s operational performance, including in relation to environmental and social aspects, with respect to precommitment work in screening, appraising, and structuring, and supervision and administration work after the Board of Executive Directors’ approval of the project and the subsequent commitment. It is rated on a four-point scale: excellent, satisfactory, partly unsatisfactory, and unsatisfactory.

Advisory Projects

development effectiveness: Synthesizes a project’s performance across five indicators: strategic relevance, output achievement, outcome achievement, impact achievement, and efficiency. It is rated on a six-point scale: highly successful, successful, mostly successful, mostly unsuccessful, unsuccessful, and highly unsuccessful.

IFC’s role and contribution: The extent to which IFC added value or made a special contribution to the advisory project. It is rated on a four-point scale: excellent, satisfactory, partly unsatisfactory, and unsatisfactory.

IFC’s overall work quality: The extent to which IFC services ensured quality at entry and supported effective implementation, through appropriate supervision and execution, toward the achievement of development objectives. IFC work quality and its two dimensions—project preparation and design, and project implementation and supervision—are rated on a four-point scale: excellent, satisfactory, partly unsatisfactory, and unsatisfactory.

Figure B1.1.2. Performance Ratings in IFC Investment and Advisory Projects

Image

Figure B1.1.2. Performance Ratings in IFC Investment and Advisory Projects

Source: Independent Evaluation Group.

Note: IFC = International Finance Corporation.

MIGA

development outcome: Measures performance across four indicators considering its impact on each affected stakeholder group: project business performance, economic sustainability, environmental and social effects, and foreign investment effects. It is rated on a six-point scale: highly successful, successful, mostly successful, mostly unsuccessful, unsuccessful, and highly unsuccessful. Until fiscal year 2019, the ratings were based on a four-point scale: excellent, satisfactory, partly unsatisfactory, and unsatisfactory.

MIGA’s effectiveness: Synthesizes MIGA’s performance across three indicators: project strategic relevance, MIGA’s role and contribution, and MIGA work quality. It is rated on a four-point scale: excellent, satisfactory, partly unsatisfactory, and unsatisfactory.

MIGA’s role and contribution: The benefits and value added that MIGA brings to the client, project, or political risk insurance industry. It is rated on a four-point scale: excellent, satisfactory, partly unsatisfactory, and unsatisfactory.

MIGA work quality: Assesses due diligence and underwriting processes, including of risk assessment and mitigation and monitoring after the issuance of the guarantee. It is rated on a four-point scale: excellent, satisfactory, partly unsatisfactory, and unsatisfactory.

Figure B1.1.3. Performance Ratings in MIGA Guarantee Projects

Image

Figure B1.1.3. Performance Ratings in MIGA Guarantee Projects

Source: Independent Evaluation Group.

Note: MIGA = Multilateral Investment Guarantee Agency.

We carried out in-depth analyses to answer the second and third evaluation questions. For the World Bank, the in-depth analyses covered all investment project financing (IPF) operations that closed between March 2020 and April 2022 (273 projects) and that had IEG ratings as of December 2022. These projects were approved between 2003 and 2021. The analysis compared this FY20–22 cohort (the RAP 2023 cohort) with a prepandemic cohort of 398 IPF projects that closed and completed their Implementation Completion and Results Reports (ICRs) between July 2017 and February 2020. For IFC, the in-depth analysis focused on 170 investment operations that were included in IFC’s Expanded Project Supervision Report (XPSR) program for calendar years (CY)20–22 and completed by December 2022. The RAP team then compared this RAP 2023 cohort with 265 prepandemic projects evaluated and validated as part of the XPSR program for CY17–19. For MIGA, the in-depth analysis covered 16 MIGA guarantee projects evaluated and validated by December 2022 from the Project Evaluation Report program for FY20–22.

Our methodological approach was guided by three principles: continuity, innovation, and symmetry. These principles collectively shaped our analysis and allowed for standardized comparisons of the Bank Group’s results and performance across time. At the same time, the approach incorporated innovative research elements in a balanced and consistent manner for each Bank Group institution. The differences in project evaluation and rating methodologies for each institution were accounted for in the symmetrical application of these approaches.

This RAP builds on the research from previous RAPs, thereby ensuring continuity and creating symmetry in RAP analyses. We extended the RAP 2021 analysis of development outcome types, allowing longer-term comparative analysis across the FY12–14, FY17–20, and FY20–22 periods.3 Moreover, we deepened the inquiry started in RAP 2021 by linking the achievement of specific project outcomes to individual project development outcome ratings. This added value to previous analyses because IFC’s and MIGA’s project development outcomes are rated at the project and subdimension levels but not at the specific outcome level. This is also the first RAP in which all projects took place, at least partly, during the pandemic. As such, this RAP expands on the limited findings of previous RAPs on the impact of COVID-19 on the Bank Group’s results and performance.

This RAP also introduced several innovative analyses that apply similar methods in a balanced and consistent manner across Bank Group institutions. First, we conducted content analyses of World Bank projects’ self-reported implementation factors and of IFC investment and MIGA guarantee projects’ key performance factors, especially within the context of COVID-19. For World Bank projects, an adapted version of the DeCODE (Delivery Challenges in Operations for Development Effectiveness) taxonomy was used to classify the implementation factors, whereas for IFC investment and MIGA guarantee projects, the RAP team leveraged the Project Insights taxonomy of performance factors developed by IEG. Second, we used supervised machine learning exercises to analyze the factors affecting World Bank and IFC investment projects’ implementation and performance over previous years. Third, we explored the reasons for low efficacy ratings in World Bank projects by using novel data from the Implementation Completion and Results Report Review (ICRR) system. Specifically, this analysis aimed to determine whether lower efficacy ratings are indicative of a failure to achieve well-defined indicators or a failure in providing sufficient evidence. Fourth, we investigated for the first time the impact of the COVID-19 pandemic on World Bank project restructuring patterns.

The report is organized into five chapters. Chapter 1 is this introduction. Chapter 2 focuses on the results and performance of the World Bank, chapter 3 on the results and performance of IFC, and chapter 4 on the results and performance of MIGA. Each of these three chapters looks at the exposure of projects to COVID-19, describes the presence of a sample selection bias, examines project performance rating trends, analyzes factors affecting project implementation and performance, explores the evolution of intended development outcomes, and assesses the validity of results framework indicators in measuring these outcomes. Chapter 2 also explores World Bank projects’ restructuring patterns. Chapter 5 provides concluding remarks for each institution and future directions to potentially take. These chapters are complemented by a set of appendixes that provide additional data and supporting information.

  1. The ratings analysis in the Results and Performance of the World Bank Group 2023 report is based on the independent ratings of the Independent Evaluation Group, unless otherwise specified.
  2. The evaluation methodology for development policy financing projects has changed. The new methodology was developed starting in mid-2020 and finalized in June 2022. See table A.1 for a comparison of the old and new methodologies of the Implementation Completion and Results Report Review.
  3. The project outcome-type data that were created by Results and Performance of the World Bank Group 2021 and used for comparative analysis only include projects closed in fiscal years (FY)12–14 and FY17–20 (second quarter).