Back to cover

The World Bank Group Outcome Orientation at the Country Level

Management Response

Management of the World Bank Group institutions would like to thank the Independent Evaluation Group (IEG) for its report, The World Bank Group Outcome Orientation at the Country Level. Management welcomes IEG’s positive findings that the Bank Group’s country-level objectives are indeed outcome oriented and articulate well the Bank Group’s contributions to achieving targeted outcomes. Management also welcomes opportunities to explore how it may improve the links between Bank Group interventions and higher-level outcomes and development goals.

World Bank Management Response

Management is pleased with the report’s conclusion that the model of how the Bank Group aims for outcomes in its client countries is sound and that teams, for the most part, practice it well. Management is encouraged by the assessment that in the Country Partnership Frameworks (CPFs) reviewed by this evaluation, “nearly all objectives were framed in terms of outcomes,” and “country teams understand well the country outcomes they are working toward” (8). Management also notes that “country teams articulate well how the combination of Bank Group operations aim to influence certain outcomes” (14). These practices have proactively been encouraged by staff training, learning activities and guidance on results and the country engagement model.

Management shares the report’s view that continuous effort is needed to better articulate indirect pathways to country outcomes in country strategy documents and is taking steps in that direction. These pathways are the Bank Group’s most distinct contribution to development results in most client segments. Management is aware that in an ever-changing development landscape, the Bank Group must clearly articulate how it combines financing, technical, and country expertise to tackle protracted challenges and respond to international commitments. In a paper presented to the Committee on Development Effectiveness in August 2020, management presented four development pathways to enhance the World Bank’s outcome orientation, one of which comprises concrete steps to enhancing the connection of World Bank–supported results to higher-level goals. Specifically, the World Bank is expanding the use of a theory of change (and links to the results framework) to improve its focus on achieving outcomes at the country level while also connecting those outcomes to higher-level goals articulated in the 2030 Agenda for Sustainable Development. Germane findings of this evaluation will be used to reinforce the proposed approach.

Management acknowledges the report’s view that, as the center of the country-level results architecture, the results framework has shown limitations, and management is willing to explore ways to strengthen the framework’s usefulness and outcome focus. While sharing the report’s view of the results frameworks’ limitations in terms of quantitative metrics, focus on attribution, and arbitrary time-boundedness anchored to World Bank processes, management believes that they have still played an instrumental role in instilling a more robust culture of results in both country teams and client counterparts. As a signatory of the Paris Declaration on Aid Effectiveness, the Accra Agenda for Action, and the Busan Partnership for Effective Development Cooperation, the World Bank has endeavored to develop results frameworks that are aligned with countries’ own results frameworks to promote mutual accountability and enhanced management for results. Nevertheless, recognizing the need to avoid overreliance on quantitative metrics to assess progress toward development outcomes, the World Bank is piloting an effort to assess how multiple Bank Group activities, in successive CPFs, contribute to higher-level outcomes and how such learning can be integrated into the CPF process to inform future country programming.

Management is of the view that any new approach, such as the one proposed by the monitoring, evaluation and learning plan, should complement rather than replace the current results system. Management appreciates that in the spirit of the Management Action Record reform, IEG has offered a general prototype (in appendix D) of what the new proposal “might look like,” presenting a vision that “is not intended to be prescriptive” (160). In the same spirit, management is thus willing to discuss with IEG the details and practicalities of such vision to ensure a good “fit” with Bank Group operations. Once a common vision is developed, management proposes to pilot such approaches in a select number of client countries that represent different client segments. These pilots could, for example, be rolled into the above-mentioned pilots currently underway to better document how interventions over multiple CPF cycles collectively contribute to country-level outcomes against specific themes. Management notes that any potential changes made to the current results system would require IEG to make appropriate changes to the way it reviews country-level performance.

International Finance Corporation Management Comments

International Finance Corporation (IFC) management welcomes IEG’s report, The World Bank Group Outcome Orientation at the Country Level. IFC supports the finding that the current country-level results system does not fully capture the Bank Group’s contribution to country outcomes and that reliance on results frameworks premised on metrics, attribution, and time-boundedness can limit a broader assessment of the complex package of support that the Bank Group delivers. IFC agrees that the current results system does not completely answer questions about how well the Bank Group’s interventions build institutions, create markets, or shape policies and under what conditions these institutions contribute to sustained positive changes to well-being.

Although the current system has weaknesses, IFC management believes that results frameworks have played an instrumental role in instilling a culture of results in country teams and client counterparts and that this should not be lost in any new approach. In this spirit, IFC management supports partnering with the World Bank and Multilateral Investment Guarantee Agency (MIGA) to pilot a new approach, in select countries, that would better assess how interventions over multiple CPF cycles collectively contribute to country-level outcomes against specific themes. IFC management appreciates the additional ideas provided in appendix D of the report, which was added too late for management to fully digest. However, the ideas presented in the new appendix, while useful, are far from what is required to determine a new approach and developing an effective system will require a significant effort. In doing this, it is important to work closely with IEG, whose approach to CPF evaluations drives much of the current process. IFC would like to reiterate the importance of streamlining any new approach so as not to add to the burden of already stretched operational teams.

IFC management welcomes the recognition in the evaluation of IFC’s efforts to deepen IFC’s inputs into the CPF and better guide investment decisions within countries through the introduction of several new tools that enrich the Bank Group’s country engagement cycle—which IEG recognizes represents a significant departure from business as usual. As highlighted in chapter 2, these include the joint IFC and World Bank Country Private Sector Diagnostics to inform and complement the Systematic Country Diagnostic, the IFC Country Strategies to lay out strategic priorities and reinforce IFC’s position in the CPF, and a country-level business plan and its subsequent reviews. The Country Strategy articulates clear investment and development impact scenarios captured by “if-then” matrixes, which lay out critical policy barriers where reform (and potentially World Bank engagement) could unlock investment opportunities.

IFC management is pleased to note the recognition provided in the evaluation that IFC Country Strategies support increasing focus on country outcomes. It is IFC’s ongoing experience that the process of combining an assessment of country development gaps and IFC’s comparative advantage with market data and analytics and a detailed understanding of regulatory and capacity barriers enables IFC to develop a common road map of priority sectors for engagement, improving the internal coherence of IFC interventions and depth of impact by influencing which sectors IFC is active in. IFC has found that it is has been an especially useful tool in shaping business development efforts with strong development impact, and discouraging project proposals that are not well aligned with the Bank Group’s strategic priorities.

IFC management particularly appreciates the finding that aspects of IFC’s actions, are particularly advisory services, not yet fully reflected in the current CPF process. IFC is expecting to increase its efforts toward creating projects that help solve a development problem through IFC upstream activities, and inherent in the design of this approach is the expectation that some things tried might need to be set aside. In that context, IEG’s proposal that more relevant and eclectic evidence on assessment of achievements and failure is of interest and worth further consideration.

Multilateral Investment Guarantee Agency Management Comments

MIGA welcomes IEG’s report, The World Bank Group Outcome Orientation at the Country Level, and finds it useful and important. The report presents many significant findings, and MIGA appreciates IEG’s observations. MIGA thanks IEG for the productive engagement during the drafting of the report.

MIGA’s integration into Bank Group country engagement. MIGA welcomes the report’s finding that there has been significant progress in integrating MIGA teams into the country engagement process, including the preparation of Systematic Country Diagnostic and CPFs as well as Performance Learning Review and Completion and Learning Reviews. MIGA’s Risk Management Officers leverage the in-depth country knowledge acquired from the country engagement process for determining country risk ratings, a key input for MIGA operational work. MIGA values highly the knowledge sharing and network building across the Bank Group that emanates from the country engagement process and has recently added additional staff to contribute to the effort. MIGA agrees with the report’s finding that the Agency prioritizes its involvement in countries that are of strategic importance, primarily International Development Association and fragility, conflict, and violence countries.

MIGA notes that the World Bank, IFC and MIGA are working closely together in several areas where synergies and complementarities are being actively pursued, leveraging each other’s strengths. In operations, the institutions are complementing and integrating their work in creating markets and fostering cross-border investment, and in overcoming obstacles that block private sector solutions. As part of this effort, the three institutions are better coordinating analytical and advisory deliverables, especially for upstream work that can greatly enhance MIGA’s ability to play a greater role downstream in support of private sector clients.

MIGA is also expanding its efforts to participate in IFC’s Country Private Sector Diagnostic, which informs and complements the Systematic Country Diagnostic, as well as IFC Country Strategies, which will also influence MIGA’s strategic choices at the country level given the often close collaboration between IFC and MIGA at the sectoral level within individual countries. MIGA’s deepening partnership with the World Bank and IFC are complementary and will promote stronger joint program delivery with the ultimate goal of delivering greater development impact than would otherwise have been the case.

MIGA also uses sector frameworks such as the Financial Sector Assessment Programs and Infrastructure Sector Assessment Programs to collaborate with the Bank Group (and the International Monetary Fund when relevant) and identify program and project opportunities. Overall, MIGA is actively engaged in all of the countries for which a Bank Group country engagement process is undertaken. MIGA works to both learn and actively contribute across the Bank Group country engagement process, as MIGA’s mandate, like that of the Bank Group, is to serve all clients—a mandate MIGA takes seriously.

Client demand and country program design. MIGA agrees with the report’s finding that client demand, including that of MIGA clients, has been a key factor in country program design. The report found that in certain contexts, the heterogeneity of client demand significantly constrains country teams’ ability for selective and sustained contribution to country outcomes. MIGA provides guarantees to private sector clients against noncommercial risks. Therefore, MIGA is typically not in a position to know in detail the specific clients or projects that it will be supporting over the medium term. Nonetheless, the role that MIGA can play in supporting development outcomes, as articulated in the various country engagement deliverables, has been clarified in recent years by adoption of the Cascade approach across the Bank Group. The Cascade approach inscribes a key and pivotal role for MIGA in signaling that the Bank Group should first seek to mobilize private sector solutions, applying official and public resources only where market solutions are not possible, including through the application of sector reforms and risk mitigations instruments, including MIGA guarantees.

MIGA contributions to country outcomes. The report finds that MIGA’s contributions to outcomes are not well captured in CPF results frameworks, as evidenced by the disconnect between stated MIGA contributions to CPF objectives and actual results indicators that capture MIGA contributions. The report recognizes that the disconnect is explained in part by the fact that MIGA’s project pipeline is largely unknown at the CPF design stage, which makes it difficult for task teams to establish indicators and targets that incorporate MIGA projects. MIGA agrees with the findings and notes the inherent challenges for the CPF results framework in capturing the distinct and joint contributions of the three Bank Group institutions—World Bank IFC, and MIGA—to country outcomes, given the unique mandates and purposes of the Bank Group institutions and the attendant business models.

Development impacts and financial returns. The report notes that IMPACT (Impact Measurement and Project Assessment Comparison Tool), MIGA’s ex ante development impact assessment framework, assesses the ex ante development effects of MIGA guarantee projects, including their ability to attract and mobilize foreign direct investments. The Rreport also notes that IMPACT has been designed to further encourage staff to prioritize potential development impacts as well as financial returns. In addition, MIGA notes that IMPACT was developed to provide a standardized methodology for assessing and communicating the potential development impact of proposed MIGA guarantee projects. The objectives include the following: (i) inform project prioritization; (ii) compare expected development impacts across the portfolio; (iii) standardize communications to relevant stakeholders; and (iv) serve as an input for MIGA’s project assessment along with other criteria. Regarding the aspect of designing IMPACT to further encourage staff to prioritize development impact as well as financial returns, the approach is consistent with MIGA’s mandate to serve all client countries and remains vital for its own financial sustainability as a development institution. MIGA will continue to support projects across the country-income spectrum, while continuing to focus on the poorest and most fragile countries, consistent with the vision outlined in its FY21–23 Strategy.

Country-level results systems and Bank Group contribution to country outcomes. The report finds that the Bank Group approach for country outcomes is sound, with country teams practicing it well for the most part. However, the country-level results system does not capture the Bank Group’s contribution to country outcomes as well as it might because it relies on results frameworks premised on metrics, attribution, and time-boundedness that do not fit well the nature of country programs. While agreeing broadly with these conclusions, MIGA underscores the fact that country-level results systems are primarily focused on accountability. Results measurement systems and approaches have progressed steadily over time, across the three Bank Group institutions. Even so, the Bank Group has recognized the inherent limitations of existing results measurement systems from the standpoint of measuring higher-order results—outcomes and impacts—that cannot be achieved through a single activity. Further, country-level results depend on contributions from multiple interventions and a broad spectrum of partners and sources of finance or advice. With respect to the specific IEG recommendation of developing an alternative country-level results system through the monitoring, evaluation, and learning plan, MIGA is of the view that the plan could be piloted for understanding better MIGA’s contributions to country outcomes.