Management of the World Bank Group welcomes the Independent Evaluation Group (IEG) report Results and Performance of the World Bank Group 2024: Managing Results in an Uncertain World and thanks the team for addressing the comments provided. This marks the 14th annual Results and Performance of the World Bank Group (RAP) report and covers a period marked by high levels of uncertainty, external shocks, and increasingly risky contexts for the Bank Group’s work. Management welcomes the report’s recognition of the Bank Group’s commitment to prioritizing and allocating more resources to low-income countries and countries classified as fragile and conflict-affected situations (FCS) and responding proactively to domestic and external shocks, particularly during the COVID-19 pandemic and ongoing global crises. The insights provided by the report are valuable for improving project preparation and supporting adaptive management during implementation.

World Bank Group Overall Comments

Management underscores the Bank Group’s progress in strengthening operational effectiveness, noting that further efforts are underway to enhance development outcomes. Management welcomes the report’s findings, particularly the improvement in development outcome ratings, which rose from 68 percent of country programs rated moderately satisfactory or above in FY 2013 to 78 percent in FY20, surpassing the 70 percent corporate target set in the Corporate Scorecard for FY19–23. Management notes that reforms are underway in the Country Engagement Framework, along with the launch of the new Bank Group Scorecard to drive further improvement and selectivity in the Country Partnership Framework (CPF). Greater selectivity in CPF objectives is expected through greater alignment with the new Bank Group Scorecard outcome areas, Bank Group programming will be further integrated, proactive risk mitigation will be strengthened through annual business planning, and the quality of CPF results frameworks will be improved through alignment with the new Bank Group Scorecard. Management’s efforts to revamp country advisory services and analytics products, such as the Country Growth and Jobs Reports, Country Private Sector Diagnostics, and Country Climate and Development Reports, will help inform Bank Group programming, using all Bank Group instruments, including the International Finance Corporation (IFC) and the Multilateral Investment Guarantee Agency (MIGA), and upstream and advisory services.

Management acknowledges the report’s finding that challenging operational environments may impede improvements in the Bank Group’s performance and outcome ratings and is committed to continuing to prioritize efforts to mitigate these risks and strengthen development outcomes. The report highlights an emerging gap between outcome ratings and Bank Group performance ratings for FCS contexts, notably for projects with extended exposure times to the pandemic and other compounding shocks in the challenging global environment. Management notes that these observed results likely reflect an interaction effect of both FCS and support for crisis response since managing for results was more complex in FCS countries during the COVID-19 period. While appreciative of the report’s insights and conclusions, management notes that some of them may be based on small sample sizes that can be influenced by minor percentage changes within the margin of error. The Bank Group is committed to improving monitoring and evaluation ratings, and management is pleased to report that proactive measures have been taken, which include refining monitoring and evaluation methodologies, revising indicators, adjusting targets through restructuring, and collecting additional evidence on project achievements.

Management welcomes the notable improvement in collaboration among the World Bank, IFC, and MIGA over the past decade and is committed to further enhancing synergy and impact. Stronger collaboration across the Bank Group will be driven by a shared view of strategic priorities in client countries in alignment with Bank Group corporate priorities and systematic efforts to leverage each other’s capacities across CPF periods and in multiple sectors. The One World Bank Group approach is also central to country programming, diagnostics, and operations to strategically determine when to tap into public or private sector solutions—or both.

World Bank Management Comments

Management appreciates the report’s finding that despite the challenging global environment of 2020–23, the World Bank successfully maintained stable overall ratings for its operations and monitoring and evaluation (M&E) with the share of operations validated moderately satisfactory or above increasing to 84 percent. The report finds a sustained upward trend in Bank performance ratings, rising from an average of 3.8 (out of 6) in FY13 to 4.3 in FY23, plateauing at 4.3 during the COVID-19 pandemic and global crises (FY20–23). The report primarily attributes the stability in the ratings to a 6 percent increase in the proportion of projects with full or partial exposure to FCS—historically associated with lower average ratings for much of the period covered—amplifying their impact on the overall World Bank average outcome rating. Management notes that the pandemic, restricted movements, and limited monitoring capabilities and evaluation efforts potentially affected Bank Group performance ratings. Management also wishes to emphasize the importance of the World Bank’s adaptive risk management approach and the significant efforts made to maintain portfolio performance despite these complex challenges, which demand more targeted, flexible, and responsive approaches, as evidenced by a realignment of 60 percent of its portfolio. Management notes that IEG’s identification strategy does not allow for empirically isolating the effects of COVID-19 and disentangling the negative effects from the mitigating effects, inferring that the negative shock from the pandemic and the mitigating effects of the World Bank’s operations would have had a net zero effect on World Bank performance. In a counterfactual scenario without the pandemic and global shocks, the positive pre-pandemic trend in year-on-year improvements in overall performance and M&E ratings would likely have continued.

Management acknowledges IEG’s insightful observation of the association among proactive risk identification, mitigation measures, and improved outcome ratings. The report’s finding underscores the validity of management’s commitment to strengthen adaptive management practices through several promising avenues, including proactively restructuring operations, rolling out the crisis preparedness and response tool kit, scaling up programmatic approaches under the Global Challenge Programs and multiphase approaches, and accelerating the implementation of effective operations. The report’s findings support management’s plans to continue strengthening adaptative risk management practices, including the calibration of the Systematic Operations Risk-Rating Tool risk ratings based on updated risk assessments and implementation of risk mitigation measures during project implementation. Management is also planning on adopting agile and innovative approaches using new technologies to collect and analyze emerging risk data to inform timely course correction.

Management recognizes the report’s finding that client capacity challenges are linked to lower performance, particularly in FCS, International Development Association (IDA), and blend countries. Simplified design can improve performance and should feature interventions aiming at building client capacity and adequate identification and mitigation of institutional capacity risks. Management reaffirms that several initiatives have been taken to address the client’s institutional capacity gaps. The World Bank’s strategic decision to expand in FCS contexts has been coupled with specific initiatives under the fragility, conflict, and violence strategy that are designed to build capacity among government agencies and partners and improve their effectiveness. The suite of tools emphasized in FCS contexts includes the Geo-Enabling Initiative for Monitoring and Supervision, hands-on expanded implementation support, World Bank–facilitated procurement, and others. The Geo-Enabling Initiative for Monitoring and Supervision builds capacity in digital data collection and analysis among government agencies and partners. New approaches such as hands-on expanded implementation support and World Bank–facilitated procurement directly assist borrowers in their procurement efforts when they are faced with capacity, market, or supply obstacles (both hands-on expanded implementation support and World Bank–facilitated procurement proved especially effective as part of the World Bank’s COVID-19 pandemic response). Third-party monitoring and implementation arrangements, partnerships including with United Nations agencies and regional organizations, and increased field presence and staff facetime will further support client capacity in FCS. Management has recently launched a new Client Capacity Program to address project implementation units’ operational learning needs during portfolio implementation. The Change Management Program for Procurement also includes a strong focus on client capacity and training.

Management welcomes IEG’s recognition of the doubling in the share of projects with M&E quality ratings of substantial or above, rising from 29 percent in FY13 to 64 percent in FY23. While recognizing the need for further work to improve the remaining one-third of projects that fall below the “substantial” threshold, management believes that the promising trend observed over the last 10 years, including at the CPF level, indicates that such gains can be maintained. Management agrees that further achievements would be facilitated by the continued provision of enhanced staff training and resources to improve M&E capacity, improvements in results frameworks and indicator selection, and enhancements in data analysis. Management concurs that continued progress in improving M&E systems, including moving results frameworks from inputs and outputs to outcomes, will be critical for the achievement of World Bank objectives and for providing adequate data for the new Bank Group Scorecard.

Management acknowledges the report finding that projects with exceptionally long preparation times (exceeding the 90th percentile) tend to underperform, noting that long lag times are often due to a lack of client buy-in or overly complex project designs. Substantial efforts are underway to shorten average project preparation time, using risk-based approaches to target more resources on more complex and high-risk operations. The report indicates that both undermine the ability of projects to achieve their objectives. Management welcomes further consultation as it deepens its analysis of the complex relationships among preparation time, design quality, and implementation readiness and refines its nuanced approach to preparation time, considering factors such as project complexity, country and sector context, and political stability.

International Finance Corporation Management Comments

Management notes positively the improving trends in the medium term, particularly given the challenging external context of recent years, while acknowledging a slight decline over the long term. Development outcomes for IFC investment projects improved by 10 percentage points over the medium term from 41 percent to 51 percent, while declining 2 percentage points over the long term. The improvement over the past five years is encouraging, particularly in Africa, which also saw the largest increase in the share of active portfolio and where operating conditions are often challenging. Furthermore, management believes that learning from positive trends is as important as learning from negative trends and found IEG’s identification of IFC investment subgroups whose performance improved over the medium term, including Disruptive Technologies and Funds, which saw a 34 percentage point increase in the development outcome rating from 14 percent to 48 percent, helpful. While in-depth analysis to identify drivers of performance over the medium term is beyond the scope of this year’s RAP, IFC management would welcome for future RAPs to examine the drivers of medium-term performance.

Management acknowledges the weak outcome ratings for IFC investments in FCS, IDA, and blend countries, where factors outside of IFC’s control continue to be key drivers of performance. Achieving high development impact in projects in FCS and IDA-eligible FCS countries remains challenging, including due to economic issues, increased civil unrest and fragility, and associated business risk. However, management’s proactive risk management and capacity-building efforts have allowed it to maintain a critical presence in these regions and play a countercyclical role at a time when foreign direct investment flows to these markets saw substantial decreases. Management notes that IFC’s upstream and advisory services have been instrumental in improving project design, increasing private sector capacity, and fostering sustainable market outcomes, particularly in IDA and FCS countries. IFC has refined its approach in FCS, with upstream and advisory work tailored to highly fragile markets. Through initiatives such as FCS Africa’s Local Champions Initiative, IFC identifies and supports potential clients in areas such as financial management and environmental and social compliance to help build a pipeline of bankable projects in FCS. For Middle East and Pakistan (the region after Africa with most FCS countries), an IDA-FCS Investment Accelerator program is being developed. The program will coordinate all IFC’s relevant tools and advisory products (environmental and social advisory, corporate governance advisory, feasibility support, among others), under a single platform to address the key hurdles, helping develop a sustainable pipeline of investable projects and sponsors. IFC is also systematically scaling up its conflict sensitivity interventions to better work in these markets.

Management acknowledges the improvements in development outcomes in Europe and South Asia and the decline in investment project ratings in the remaining regions over the long term. Latin America and the Caribbean contributed to the decrease in IFC-wide development outcomes more than other regions due to its large share of reviewed projects (25 percent) and the decline in its ratings over the long term (from 60 percent of projects rated mostly successful or better to 47 percent). This is tied to a strategic shift in upper-middle-income countries in the Latin America and the Caribbean region, where IFC is taking more risks to innovate and test new business models by developing new products and markets and working with new clients. Management thanks IEG for bringing to our attention the gaps in the Latin America and the Caribbean region’s work quality that need improvement. Lessons highlighted will serve to instruct teams to continue testing similar business models and to develop stronger risk mitigants for future projects.

Management welcomes IEG’s plan to develop a taxonomy of factors linked to the development effectiveness of IFC advisory services for future studies. Unlike for IFC investment projects, there is no typology in IEG for factors linked to performance for IFC advisory services projects at this point. This limits the analysis of challenges that can be conducted and thus management’s understanding of levers we can employ to drive development effectiveness, after controlling for external factors. Management would thus find the taxonomy helpful in promoting learning on the effectiveness of future advisory services engagements and understanding what makes for high-quality preparation in IFC advisory services projects.

Management finds the analysis on challenges and levers insightful and relevant and welcomes the additional preliminary analysis that IEG has conducted on work quality, too. These analyses offer a helpful framework to identify areas that can influence development results, including those within and outside management’s control. This is particularly relevant as IFC continues to focus on more challenging markets where issues around client quality, risks, and lack of market information make employing available levers more challenging. Overall, IFC is providing capacity building in these operating environments to expand the pool of high-quality clients. As we continue to focus on more challenging markets, it would be helpful for IEG and Bank Group management to reflect more generally on how the outcomes and performance should be measured across time when the risk environment has fundamentally changed, and there is an expectation for the institution to take more risks. It is important to note that not all challenges and risks—even if noted on the front end—can be mitigated.

Management continues to strengthen our capabilities for increased tracking of outcomes through enhancements to the Anticipated Impact Measurement and Monitoring (AIMM) system, and we find that IEG’s gap assessment is misaligned with the AIMM framework and approach. In FY24, management launched the AIMM Navigator, a technology platform that allows IFC to track development impact more systematically and consistently within the AIMM framework. IFC tracks outcome indicators related specifically to impact claims and those that are needed for corporate reporting purposes. IFC intends to track only the most significant indicators that underpin the AIMM assessments and are reflected in the Development Impact Indicator table, which is included in the Board paper. In contrast, IEG’s typology, developed for RAP 2021, tries to cover all potentially relevant outcomes included in the AIMM narratives, driving the reported gap in outcome tracking.

Multilateral Investment Guarantee Agency Management Comments

MIGA notes RAP 2024’s confirmation that MIGA has maintained a consistent trend in development outcome ratings, between 65 percent and 70 percent. Following a constructive and evidence-based dialogue with IEG colleagues, both parties agree that the one percent decline in development outcome ratings over the long term is not statistically significant at the MIGA-wide level. However, MIGA also recognizes the challenges in interpreting the ratings due to the delay in completing the current self-evaluation program.

MIGA acknowledges IEG’s findings on the lower outcome ratings among projects in IDA and blend countries. These ratings indicate challenging environments in these countries in achieving expected development outcomes. MIGA noted IEG’s additional analysis, which highlighted that key factors affecting outcomes, such as cost overruns, construction delays, and legal and regulatory risks, are beyond MIGA’s control.

MIGA emphasizes its strong record in monitoring and managing project risks throughout the life of the guarantee contracts. This involves monitoring project and country risks, development indicators, and environmental and social risks. MIGA plays a proactive role as an honest broker in helping resolve emerging contractual or country risks.

MIGA highlights that development outcome ratings for fragility, conflict, and violence–status projects over the past six-year cycle are at 75 percent, surpassing MIGA’s overall average. For the last decade, fragility, conflict, and violence–status projects have consistently achieved strong development outcome ratings, reflecting MIGA’s effectiveness in delivering development impact in challenging environments. MIGA continues to prioritize this area, leveraging lessons learned to enhance development outcomes.

MIGA acknowledges IEG’s observation regarding the realization of foreign investment effects, which extend beyond direct project benefits. It is important to note that these evaluations took place before the introduction of the ex ante development impact assessment tool. MIGA had already been extensively focused on specifying and assessing foreign investment effects. These experiences led to improvements that culminated in the Impact Measurement and Project Assessment Comparison Tool framework, MIGA’s ex ante development outcome assessment tool, piloted in FY19 and fully launched in FY20. The recent introduction of the Bank Group Scorecard, which features a stronger outcome focus, has also helped us sharpen our attention in development impact assessment and the tracking of such effects, especially those aligned with higher levels of outcomes. Furthermore, the launch of the Bank Group guarantee platform enhances the collaboration across Bank Group institutions further, with opportunities for furthering country-level collective engagement that can support stronger project supervision and monitoring.

In this regard, MIGA welcomes IEG’s ongoing commitment to various lenses of outcome analysis across the Bank Group and believes that IEG can enrich data quality assurance and methodological fine-tuning for sharpening the outcome assessment.