Back to cover

Results and Performance of the World Bank Group 2022

Chapter 5 | World Bank Group Performance at the Country Program Level

Highlights

This Results and Performance of the World Bank Group found weaknesses in country program relevance with regard to selectivity (for example, Country Partnership Framework [CPF] objectives that were too numerous or too broad), adaptiveness (for example, insufficient preparedness to respond to changes in country conditions, government commitment, or Bank Group priorities), and realism (for example, operations overestimating implementation capacity and underestimating political challenges). Furthermore, CPFs and their results frameworks also often relied overwhelmingly on the lending portfolio and insufficiently integrated and leveraged advisory services and analytics and International Finance Corporation and Multilateral Investment Guarantee Agency support.

Risks are identified well up front, especially regarding macroeconomic risks and risks associated with external shocks, but they are not identified as well regarding implementation capacity and political economy risks.

A timelier and ongoing reevaluation of risks, adaptation of programs, and updating of results frameworks is needed, certainly at the time of the Performance and Learning Review. However, the reasons for adaptation are important, and potential trade-offs between adaptation and shifting goalposts need to be recognized and managed.

Risk mitigation is often partial, and the Bank Group lacks procedures at the country level, such as scenario planning, for acknowledging risk that cannot be fully mitigated.

Completion and Learning Review (CLR) Reviews provide limited evidence on the quality of implementation support, including in middle-income countries, where challenges are emerging at the subnational level.

The potential for a One Bank Group approach has not been fully realized, and the contributions of the International Finance Corporation and the Multilateral Investment Guarantee Agency have been captured poorly within CPFs.

Partnership with donors is seen as a Bank Group strength, especially in low-income countries.

World Bank Group (2021c) and World Bank (2020a) pointed to weaknesses in the Bank Group’s approach to country programs.1 The Bank Group subsequently revised its country engagement guidance in July 2021 (World Bank Group 2021a). Box 5.1 summarizes the main enhancements in that new guidance. These enhancements did not yet apply to the country programs covered by this RAP. Instead, this RAP establishes the baseline against which the country program performance resulting from the new country engagement guidance can be assessed in the future. Appendix I discusses some aspects of the new guidance that can be strengthened.

Box 5.1. Enhancements in the Revised Country Engagement Guidance of July 2021

The World Bank Group revised its country engagement guidance, country engagement procedure, and the Systematic Country Diagnostic guidance to strengthen outcome orientation in the Bank Group’s approach to country engagement. The revisions were informed by ongoing discussions with the Committee on Development Effectiveness and the Independent Evaluation Group report on outcome orientation at the country level (about how to best capture Bank Group contributions to long-term country outcomes).

The revised country engagement guidance introduces high-level outcomes (HLOs) in Country Partnership Frameworks (CPFs). HLOs are defined as a sustained improvement in the well-being of the poorest and most vulnerable people—for example, their health, security, mobility, opportunity, livelihood, or standard of living. These HLOs are primarily drawn from a client country’s own development strategy and are meant to be aligned with the Sustainable Development Goals. HLOs are typically achieved over a time horizon that extends beyond a single Country Engagement Cycle, are set at a higher level than CPF objectives, and typically result from the combined effort of multiple partners.

The revised Systematic Country Diagnostic guidance now includes guidance for teams to identify long-term development outcomes and HLOs that are critical to the achievement of the twin goals and to articulate constraints and opportunities for achieving them.

During CPF preparation, Bank Group teams, in consultation with clients, will select relevant HLOs and CPF objectives for their country programs.

The updated guidance became effective on July 1, 2021, and applies to all country engagement products that have Concept Note reviews after the effective date.

Source: World Bank Group 2021a.

This chapter provides a qualitative analysis of the performance of the Bank Group at the country program level based on the RAP’s analysis of the CLR Reviews of the 50 randomly sampled countries.2 Box 5.2 describes the criteria used.

Box 5.2. Criteria for Examining the Performance of the World Bank Group at the Country Level

Country Partnership Framework (CPF) Design and Implementation

1. Relevance of country program (for example, selectivity and framing of CPF objectives, choice of interventions, adaptiveness, and realism of program design)

2. Quality of results framework

3. Risk identification

4. Risk mitigation

5. Bank Group support to implementation (including technical assistance)

6. One Bank Group approach

7. Partnership with donors

8. Additional elements (support for twin goals, “knowledge bank,” dropped or canceled projects)

Line of Sight

1. From Bank Group support to CPF objectives/development outcome

2. From CPF objectives/development outcomes to high-level outcome (which are meant to be primarily drawn from a client country’s own development strategy and are meant to be aligned with the Sustainable Development Goals)

Sources: Independent Evaluation Group; World Bank Group 2021a.

Country Partnership Framework Design and Implementation

Country Program Relevance

This RAP’s definition of relevance includes a larger set of dimensions than is traditionally the case. Relevance includes, for example, selectivity and framing of CPF objectives, choice of interventions, adaptiveness, and realism of program design. Using this definition of relevance, this RAP’s analysis found that the CLR Reviews for just 21 of the 50 sampled countries pointed to good country program relevance (figure 5.1).

Figure 5.1. World Bank Group Performance at the Country Program Level

Image
A horizontal bar chart showing Bank Group performance on criteria at the country program level.

Figure 5.1. World Bank Group Performance at the Country Program Level

Source: Independent Evaluation Group.

Note: The figure shows the number of CLR Reviews. Criterion discussed means that information provided in the CLR Review addressed the specific criterion. Good performance on criterion means that information provided in the CLR Review addressed the specific criterion. CLR = Completion and Learning Review.

Framing of Country Partnership Framework Objectives

Poorly framed CPF objectives include those that are excessively broad and monitorable only infrequently or at the countrywide level (for example, increasing rural incomes). Other poorly framed CPF objectives are those that are excessively narrow, often project related and output based (for example, doubling the number of rural microloans). Some objectives are defined in ways that fail to address the highest needs in that area.

The Sri Lanka CPF (FY13–16) had 18 objectives, some of which could have been combined. For example, objective 7 (increased connectivity) focused exclusively on road connections, thereby duplicating objective 12 (improved quality and sustainability of roads). In the Tajikistan CPF (FY15–18), the results framework set appropriate targets and baselines, but objectives were framed poorly, providing little support for implementation and results monitoring. According to the CLR Review, most, if not all, of the objectives were pitched either at the project level (for example, objectives 8, 11, and 19) or in the form of indicators (for example, objectives 1, 2, 3, 5, 6, 7, 10, 12, 13, 14, 16, 17, 20, and 21). In the São Tomé and Príncipe CPF (FY06–09), for instance, two outputs (number of donor meetings and a new development strategy submitted to the National Assembly) were the stated CPF objectives. In Senegal, a 2014–15 enterprise survey found that tax rates and tax administration were the primary constraints for firms, but the CPF objective to enhance the investment climate did not reflect this.

Choice of Interventions

As indicated by the CLR Reviews that the RAP examined, sometimes the Bank Group’s choice of interventions was not adequate to achieve the stated program objectives. In the São Tomé and Príncipe CPF (FY06–09), a microcredit feasibility study was not a sufficient intervention for strengthening financial intermediation and broadening access to credit. In the Sri Lanka CPF, the activities proposed by the Bank Group to enhance accountability and transparency of public funds alone were not adequate support for the government’s development goal of increasing fiscal space and increasing efficiency of public spending by increasing revenue and lowering expenditures. And in the Mauritius CPF (FY07–15), 3 of 21 objectives were not sufficiently supported by lending or nonlending activities.

The Brazil CPE pointed to the need for Bank Group support to be more catalytic, meaning that programs and projects should have an impact extending beyond any one intervention. Examples of a catalytic response include support for reforms that create enabling environments and incentives for other actors, activities to enhance demonstration effects and replicate positive results, and engagements that leverage the Bank Group’s knowledge base and its convening role to facilitate cross-sectoral dialogue. However, the 2017 CLR Review noted that the catalytic effect was blunted somewhat by the sheer size of the country, which led the Bank Group to spread its resources too thinly across 20 subnational borrowers plus the federal government.

In Mexico, the CPE (World Bank 2018) found that the Bank Group needed to engage more fully with lagging regions, a recommendation that applies to all middle-income countries with large regional income disparities and pockets of extreme poverty. Although there may be obstacles to subnational lending, there is often scope for engaging effectively through advisory work and more effective joint programing among IBRD, IFC, and MIGA.

Selectivity of Objectives

When programs have too many objectives, implementation may be compromised. In the Kosovo CPF, the program had 26 outcomes and 31 indicators and was far too ambitious relative to the size of the IDA envelope. Objective 5 (promoting sustainable employment and social inclusion) was too compound in nature, lacking the focus needed to guide interventions. In Madagascar, the number of CPF objectives and outcomes needed to be reduced when projects were canceled, but no adjustment was made. In the Romania CPF, the number of focus areas remained unchanged throughout the program, but the number of objectives increased from 7 to 14, with the number of associated indicators rising from 16 to 43. The Argentina CPF (FY15–18) included a compound four-in-one objective to increase access to electricity, safe drinking water, housing and sanitation supported by a wide range of interventions. Three of four areas were not achieved under this objective, with only a narrow housing pilot achieved during CPF implementation.

The Rwanda CPF is a good example of selectivity. With just 12 well-defined objectives, the program was sufficiently selective and played to the Bank Group’s strengths in supporting infrastructure, agriculture, and governance. In general, CPFs with too many objectives (typically more than 12 objectives or with multiple compound objectives) tended toward poorer development outcomes than CPFs that were more selective.

Factoring in Flexibility and Adaptation

An a priori assessment of government commitment to the planned Bank Group program is an essential aspect of CPF design. Nonetheless, during CPF implementation, government commitment can change, and governments can also change. Fiscal space and debt burdens evolve, affecting attitudes toward planned Bank Group lending or choice of lending instruments. When circumstances change, Bank Group teams need to respond flexibly and adapt appropriately, adjusting the CPF program and the underlying results framework as necessary. Such adaptation may not be without its challenges given that multiple stakeholders may need to be involved within the client country. Yet adaptation is critical for maintaining the relevance of the country program. Furthermore, at the country program level, potential trade-offs between adaptation and shifting goalposts need to be recognized and managed.

This RAP’s review of 50 sampled countries’ CLR Reviews suggests two interrelated problems: failure to adjust the program sufficiently and in a timely manner to respond to changing circumstances, and failure to revise the CPF results framework to keep pace with changes in the Bank Group program. The first problem reduces the program’s relevance and effectiveness. The second problem leads to a missed opportunity to report the Bank Group contribution stemming from changes to the program and for learning.

In the Bhutan Country Partnership Strategy (CPS), the government ultimately decided against planned investment projects and P4R operations, opting instead for a smaller number of policy-based operations. In value terms, only about half of the planned IDA lending program was delivered through policy-based operations. The shift to quick-disbursing policy loans, without more traditional projects, weakened efforts at longer-term institutional capacity building and state modernization. The turn away from lending shifted Bank Group efforts into a large volume of ASA (45 products delivered during the CPF period versus only 4 loans), plus trust-funded technical assistance. The high volume of ASA and technical assistance only partially compensated for the longer-term capacity building designed into project lending and challenged absorptive capacity—the CLR Review in Bhutan described the ASA program as being too large and dispersed among sectors, given the country’s limited absorptive capacity. Furthermore, the shift toward policy-based operations and ASA was not fully addressed at the Mid-Term Review of the CPF, with only marginal adjustment of the initial objectives and indicators that were rooted in the planned investment lending. As a result, 4 of the 7 program objectives were only partially achieved at the end of the CPF period.

The Bolivia CPF is an example of where the lending program that supported 16 CPF objectives was only half delivered. Expected CPF development outcomes and indicators were not adjusted sufficiently at midterm to compensate for the lending shortfall and lack of projects in key areas of the Bank Group program. As a result, 8 of the 16 objectives were not achieved, partially achieved, or not verifiable at the end of the CPF period, and the overall development outcome was deemed moderately unsatisfactory.

Quality of Results Frameworks

The performance of the Bank Group was weakest in the quality of the results frameworks underpinning the CPFs, as also indicated by World Bank (2020b). In many results frameworks, the objectives were poorly articulated (too vague, too broad, or too narrowly project related) and not accompanied by adequate indicators that could assess the achievement of these objectives and provide a link to the program’s interventions. This RAP’s analysis found that the CLR Reviews for 4 of the 50 sampled countries had a good results framework (figure 5.1).

A significant weakness in CPF results frameworks was the inappropriate choice of indicators. Failings included (i) indicators that were nationwide in scope (too broad to assess the Bank Group’s contribution); (ii) indicators that could not be easily quantified; (iii) indicators that were irrelevant for measuring progress toward the desired outcome; (iv) indicators that measured inputs, outputs, or intermediate outcomes rather than the relevant program outcome; (v) indicators that were not available or not verifiable at ex post evaluation of the CPF; (vi) indicators that were not generated by Bank Group programs if not otherwise done by government; (vii) indicators that had never been tracked by the country; and (viii) importantly, indicators reported individually when a set of indicators needed to be considered together to determine the achievement of a CPF objective.

In the Senegal CPF, for instance, one indicator was the increase in commercial cases tried by courts—not necessarily the best measure of progress toward the objective of an improved investment climate. Milestones or output measures are often inappropriately substituted for outcome measures. The South Africa CPF is a case in point: the results framework contained 73 milestones and outputs but few outcome indicators with baselines and targets. In the Ukraine CPF, there were no indicators to track the contribution of project results to country outcomes such as control of corruption, business environment, agricultural productivity, access to finance, and financial stability.

In the Bhutan CPS, IEG found that for four of the seven objectives, the chosen indicators were not relevant to measure progress toward the stated objective. Indicators on revenue generation and the size of fiscal deficit were an insufficient measure of progress toward the objective of strengthening fiscal efficiency. Indicators on the volume of foreign direct investment were poorly defined and not directly relevant to the objective of improving the regulatory framework.

In the Brazil CPF, the indicators for early childhood, primary and secondary education, and for access to water and sanitation, were all nationwide in scope and thus too broad to capture the contributions made by Bank Group interventions. Also, the chosen agriculture sector indicators (number of producers supported and number of states promoting climate-smart agriculture) were not relevant for monitoring progress toward the stated objective of enhancing market access and boosting small farmers’ adoption of climate-smart farming practices.

Risk Identification Handled Well

This RAP’s analysis found that CLR Reviews for 32 of the 50 sampled countries show good risk identification (figure 5.1). In Senegal, limited government capacity was correctly identified as a major impediment to program implementation. Institutional capacity constraints were also well anticipated in the CPFs for the Philippines, Rwanda, and São Tomé and Príncipe.

Most CPFs did a good job of identifying risks, especially macroeconomic risks and risks associated with external shocks, but they were less adept at assessing risks from limited implementation capacity and lack of local commitment and were overoptimistic in assessing political risks.3 These risks can be particularly detrimental to achieving policy reform in a country and should also be addressed at the country program level rather than just at the project or other levels.

Uganda is a case in point. Eight of the 12 objectives of the CPF (FY11–15) were either partially achieved or not achieved, suggesting that the country program design was based on an overoptimistic reading of Uganda’s institutional capacity. In Sri Lanka, the CPF overestimated the government’s willingness to carry out policy reforms. The World Bank underestimated the political economy limitations that shaped the Marshall Islands’s poor commitment to reforming key areas in the program, such as in the energy and information and communication technology sectors. The CPF program in Montenegro missed the risk of insufficient government ownership, failing to anticipate the corresponding obstacles to implementing reforms in areas such as environmental and public health management. In Pakistan, the World Bank collaborated with the International Monetary Fund (IMF) on the Stand-By Arrangement that closed in September 2011, leading the policy dialogue on tax administration, electricity, and social protection issues. However, it took more than two years to reach agreement on a new IMF Extended Fund Facility Arrangement because the political risks identified at the outset did materialize, and the Stand-By Arrangement was already off track at the time of closing. The program in Romania underestimated the risks to planned lending from changes in government and administrative weaknesses. It also underestimated the extent to which the government’s focus on the European Union (EU) and its access to EU grant funds would influence the scope for Bank Group intervention.

Risk is constantly changing, but the Bank Group does not systematically update its CPF risk analysis on a timely basis. In many cases, IEG found that when country circumstances changed significantly for political, economic, or other reasons, a Performance and Learning Review (PLR) was either not issued when it seemed warranted or the program was not adequately adjusted at midterm.

In Senegal, the World Bank, faced with new risks, did not use the opportunity presented at the PLR stage to revise the overambitious set of 15 objectives and 27 indicators in the CPS. In Uganda, emerging governance and development risks led to changes in the Bank Group’s portfolio but not in the Country Assistance Strategy objectives. Although the Ukraine CPS objectives remained relevant throughout the program period, a PLR should have been completed during the time period of the program (FY14–16) as noted by the CLR Review, given the changes in the political and economic environment, the increased size and different instruments of the World Bank program, and new risks faced by the Bank Group. By contrast, the Bank Group responded flexibly in São Tomé and Príncipe, adapting the program to the uncertainties related to potential inflow of oil revenues and the government’s renewed emphasis on eliminating constraints to growth.

Risk Mitigation Handled Less Well

World Bank teams were adept at identifying risks, but they were less successful in mitigating them. This RAP’s analysis found that CLR Reviews for 48 countries discussed risk mitigation, with just 18 of those showing good risk mitigation (figure 5.1).4

The Bank Group relies on the Mid-Term Review (the PLR) to adjust the CPS to a changing risk profile. The Bank Group’s PLR guidance states that a PLR should be undertaken every two years or at the CPF’s midpoint, although there is some flexibility in the guidance to consider relevant country circumstances in deciding on the frequency and duration of the PLR (World Bank Group 2021a). However, this flexibility is not always used. For example, when government commitment to reform slackens, the Bank Group needs to adjust its pipeline, especially for policy-based operations, but the Bank Group may fail to act or act in a timely fashion.

ASA work on mitigation measures needs to be responsive and timely. In the Tajikistan CPF implementation, ASA on the political economy drivers of policy reform came too late in the program to help shape mitigation measures. Similar delays and consequences occurred in Sri Lanka: the revenue incidence analysis, the enhancing competitiveness note, and the Public Expenditure Review were delivered too late to mitigate the fiscal risks they were intended to address.

CPFs lack procedures for acknowledging known risks that cannot be fully mitigated and proposing appropriate program adjustments as these risks materialize. The response to crises as known risks materialize is often ad hoc and not adequately informed by how the Bank Group had responded across countries facing similar circumstances (appendix J). The programs in Burundi and Myanmar illustrate this shortcoming. The Burundi CPF (FY13–16) is an example of an FCS country where the Bank Group was unable to fully mitigate the effects of instability and violence resulting from the 2015 political crisis, even when they were clearly identified as a risk. If risk cannot be fully mitigated, then the Bank Group needs to think through responses and scenarios carefully. This requires thoughtful political economy analysis as a background to CPS design. The CLR also argues that continued World Bank engagement during a political crisis can enhance the relationship of trust and the World Bank’s capacity to conduct dialogue on critical development issues. Staying appropriately engaged rather than vacillating between engagement and disengagement is essential to maintaining influence on key issues related to poverty and social welfare during times of crisis.

In Myanmar’s portfolio, risks and capacity constraints are higher than in the average Bank Group–FCS client country, calling for greater-than-normal selectivity and more emphasis on capacity building. The main risk that materialized was ethnic conflict (the stalling of the peace process in 2016) and renewed violence in the Kachin, Rakhine, and Shan states. In the Myanmar CPF (FY15–19), the treatment of social inclusion and administrative capacity was uneven, undercutting efforts to respond to conflicts. Portfolio risks increased as the number of new projects increased from 9 in 2016 to 13 in 2018.

Timely political economy analysis is needed to inform the CPF design process. Program design needs to candidly acknowledge the limits of the Bank Group’s ability to mitigate these risks and consider in advance possible adjustments to the Bank Group program as risks materialize.

This lesson is also emphasized in the Tunisia CPE, which argued for greater adaptability to evolving risks, particularly political economy risks. It noted that the use of multisector development policy operations, underpinned by sound analysis, helped focus support, reinforce coordination across donors, and increase responsiveness to the government’s needs. But improved streamlining and timing of measures would have been useful, particularly by mid-2012, when the political context became more volatile. The World Bank may have overestimated the government’s commitment to reform. The CPE recommended that program design acknowledge political economy risks and provide for mitigation. It also suggested that the World Bank could have tried to build public support for reforms in the interim—before political order was restored—using its rich analytic work to better inform and build the capacity of such groups as trade unions, think tanks, civil society organizations, and parliament (World Bank 2014c).

During program implementation, the Bank Group needs to modify its ambitions and adjust its focus and timetable when risks materialize. In Poland, when most of the planned investment lending did not materialize (partly because EU grant funding substituted for World Bank loans), additional ASA were commissioned as an alternative to help achieve the stated development outcomes. The results framework was not adjusted to reflect this change. In Jamaica, the government changed midway through the program. The new administration was more supportive of the reforms that the World Bank, the IMF, and other partners advocated. The CPS was adjusted accordingly, although institutional and capacity constraints argued for a smaller program than the one approved later.

The CPE for resource-rich developing countries emphasized the need for risk mitigation in the face of volatile commodity prices. It recommended that the World Bank and the IMF maintain a dialogue on macroeconomic and fiscal policies and help countries build fiscal buffers while prices were high, remaining prepared to offer exceptional budget support in the event of a downturn.

Implementation Capacity and Implementation Support—A Relatively Neglected Area for Relieving Capacity Constraints, Including at the Subnational Level

CLR Reviews were weakest in providing evidence of Bank Group support for implementation. In two-thirds of sampled countries, it was not possible to assess the quality of support for implementation because of a paucity of evidence. Of the CLR Reviews for the 17 countries that provided evidence on this criterion, only 11 countries had good implementation capacity and implementation support (figure 5.1).

Haiti (CPF FY09–14) is a good example of strong implementation support in a low-income and FCS country. The World Bank was quick to respond to the 2010 earthquake with nonlending technical assistance, timely adjustments to the World Bank program, a damage assessment, and additional commitments for recovery and reconstruction. The Haiti CPF and Interim Strategy Notes (ISNs) before and after the devastating 2010 earthquake are a good example of strong implementation support by the Bank Group team. Especially in the aftermath of the earthquake, the Bank Group complemented its increased financial support with a high degree of technical assistance. Twenty-eight of 31 ASA tasks were technical assistance support supplemented by trust-funded technical assistance. This helped Haiti maintain strong disbursement rates and good implementation of various emergency operations in the aftermath of the earthquake, when institutional capacity was at its weakest.

Although capacity constraints tend to be better recognized within low-income and FCS contexts, the Bank Group also faces emerging challenges in middle-income countries. The need to support implementation capacity is particularly acute for subnational governments in middle-income countries aiming to decentralize and deconcentrate government. These middle-income clients often will not borrow on IBRD terms to support soft interventions such as capacity building and technical assistance, further complicating the design of lending operations at the subnational level. Bank Group support for building subnational government capacity was insufficient in the Brazil program, for example. The Bank Group responded to the government’s need for this type of support, but interventions were spread too thinly, covering 20 subnational borrowers plus the federal government. Portfolio performance lagged regional and World Bank–wide averages as the share of subnational lending in the overall portfolio rose. Implementation problems were most acute in sectors and programs focused at the subnational level, and many CPF indicators defined at the subnational level were not verifiable or not achieved. Two lessons emerge from this example: Innovative efforts are needed to identify financing for capacity building at subnational levels, and a good dialogue with and commitment from national government is necessary to work effectively at subnational levels. In Nigeria, the World Bank used four P4Rs at the state level to support institutional changes in fiscal governance, education, health, and growth. Programs under the same project worked independently as separate operations requiring 170 project implementation units—more than half the number of World Bank–supported project implementation units in the Sub-Saharan Africa Region. Although the World Bank promoted state coordination mechanisms through annual country portfolio performance reviews, further progress will require helping to reduce the number of project implementation units and facilitating synergies.

The One Bank Group Approach—Still a Work in Progress

Despite the implementation of several initiatives since IEG’s 2017 Learning Engagement, which reviewed two decades of experience with the One Bank Group approach (World Bank 2017b), the One Bank Group approach still remains a work in progress.5 The contributions of IFC and MIGA have been poorly leveraged within CPFs. This RAP’s analysis found that CLR Reviews for 45 countries discussed the One Bank Group approach criterion, but just 18 countries had good performance on this criterion (figure 5.1).

In Mexico, IEG found that the World Bank, IFC, and MIGA could work together more effectively to crowd in other sources of private and official finance for greater development impact. In Pakistan, the World Bank and IFC agreed to cooperate in several areas, but synergies were hard to attain within the Bank Group because parallel financing between IFC and the World Bank proved problematic due to different timelines and operating procedures. Collaboration between the World Bank and IFC was slow but improving. In Nicaragua, the joint IDA-IFC work that was planned in education, health services, and financial innovation failed to materialize, and the World Bank and IFC proceeded on parallel but separate tracks. In Tajikistan, the World Bank and IFC worked together on reforms to the business environment—IFC through advisory services and the World Bank through the Private Sector Competitiveness Project. It was envisioned that a series of development policy operations would help remove barriers to private sector development, opening the way for IFC to invest between $10 million and $20 million per year. But the development policy operation series was dropped due to a difficult macroeconomic environment, and the One Bank Group approach’s potential was not realized. Bank Group strategies for the resource-rich developing countries failed to convey how the World Bank and IFC might jointly promote much-needed economic diversification. The expected dividends from cooperation also failed to materialize in Armenia, Bhutan, and Brazil. World Bank teams need to calibrate the One Bank Group approach to define meaningful collaboration in countries where the policy environment is not conducive to private sector investment or expected reforms fail to materialize.

There are two related issues worth highlighting about a One Bank Group approach. The first is that collaboration and coordination across the Bank Group in the field is still a work in progress. The second is that the CPF and its accompanying results framework often fail to capture existing collaboration, coordination, and contribution by the institution’s private sector arms. This is partly a weakness in the initial CPF’s design and partly a reflection of a failure to capture unplanned IFC and MIGA opportunities that arise during CPF implementation. The Côte d’Ivoire CPF (FY10–14) is an example of where the results framework underreports on both IFC and MIGA, and the PLR does not adequately adjust to reflect an upswing in IFC and MIGA activity. IFC made 20 investments during the CPF period, representing a tenfold increase in the value of the IFC portfolio, and MIGA made four guarantees consistent with corporate efforts to engage more in FCS. Yet, as reported in the CLR Review, the results matrix says little about IFC activities and the CLR does not discuss the IFC portfolio.

In the Uganda CPF, the results framework facilitated the monitoring of progress in addressing inefficiencies in land registration through World Bank interventions but overlooked the equally important contribution of IFC’s work on simplification of licensing and online filing for taxpayers, including small and medium enterprises. The complementarity of these interventions in support of the business environment was not reflected adequately. Likewise, in Sri Lanka, IFC’s development outcome could have been better reflected in the results framework if appropriate indicators had been identified. A similar oversight was manifest in the Montenegro program. In the Pakistan CPS results framework, many indicators lacked benchmarks, particularly those related to IFC, because IFC contributions were not adequately targeted when the strategy was prepared. As recognized in the completion report, although the results framework covered IFC activities as part of the array of CPS implementation instruments, IFC input into the formulation of outcomes and milestones was not evident.

Three good examples of the One Bank Group approach hold lessons for the future. Senegal is an example of superior synergies across the Bank Group. Cooperation among the World Bank, IFC, and MIGA helped push complex reforms through in the energy sector, improve the investment climate, and strengthen financial infrastructure. The Dakar-Diamniadio Toll Road Project and the Scaling Solar Program were other instances in which the One Bank Group approach succeeded. The toll road project was the first of its kind in West Africa to involve a public-private partnership sponsored by the Bank Group. Uganda also showcased the potential of the unified approach. Support to the Bujagali Hydropower Project was provided through an IDA partial risk guarantee, an IFC loan, and a 20-year MIGA guarantee. IFC and MIGA also cooperated well in supporting Umeme, the electricity distribution company. Argentina is another example of capturing synergies across the Bank Group effectively. The Mobilizing Finance for Development approach was used to help develop a market for renewable energy. All three Bank Group institutions contributed to the program with several joint or complementary activities. Operationalizing this approach and replicating it for other relevant programs in Argentina requires systematic coordination among the Bank Group institutions and with counterparts. But according to the CLR, the prerequisites for a successful leveraging of private sector financing for renewable energy need to be spelled out so that others may replicate Argentina’s success.

Overall, IFC and MIGA need to be integrated more fully in the design and implementation of the Bank Group’s overall program. Additionally, their contribution needs to be made explicit in the results framework at the initial design stage and adjusted at the PLR stage so that the effectiveness of their support for development outcomes can be assessed properly (appendix K).

Partnering for Results—Promising Progress

The Bank Group performed well on partnering for results. This RAP’s analysis found that CLR Reviews for 47 countries discussed it, of which 31 countries showed good partnering for results (figure 5.1).

The Bank Group has coordinated its work effectively with other development partners in most countries and has played a central role in donor coordination in other countries. In Uganda, for instance, the Bank Group has served as both permanent co-chair of the former Joint Budget Support Framework (which included the 12 budget support development partners) and chair of the Local Development Partners’ Group. In Rwanda, the World Bank is also well integrated into the donor coordination system, which operates through quarterly and annual meetings, including sector working groups and development partner consultative groups. Likewise, in Tajikistan, the World Bank chairs the Development Coordination Council, which comprises 13 working groups and 29 development partners. And in Ukraine, the Bank Group coordinated its program and activities with many development partners, including the IMF, the EU, the United States Agency for International Development, and the United Nations.

Donor coordination has helped enhance the credibility and selectivity of World Bank programs (Senegal); helped reconstruction and reconciliation efforts in Mindanao Island (the Philippines); helped support reforms for a new government as part of a larger international package including World Bank policy-based operations (Ukraine); helped support two state-owned banks prepare for privatization (Serbia); brought in other donors in energy (Uganda); and resulted in jointly agreed procedures on procurement, financial management, safeguards, and project implementation support (Tajikistan). This is a solid record of achievement, but within this are many examples where donor partnership failed to materialize or was ineffective. In Afghanistan, for example, contradictory advice and competing donor programs compounded conflicting views among different government agencies, leading to a “missing middle” in the absence of agreement on subnational governance, as noted in IEG’s 2013 CPE for Afghanistan.

Additional Elements of World Bank Group Performance

Support for the Twin Goals—Greater Attention Needed, Especially in the Face of Shocks

Changing country circumstances and Bank Group responses can produce unintended consequences that may undermine the twin goals. In implementing the Uganda CPF (FY11–15), a decision was made to cancel budget support that had been a regular feature of the program for many years. The cancellation reflected concerns about transparency, accountability, and corruption. Pro-poor spending by the government subsequently contracted, squeezing social services in the most vulnerable areas. The CLR Review suggests that the Bank Group could have made more effort to protect poor people in Uganda without the budget support through better dialogue on domestic resource mobilization and pro-poor spending or through compensating mechanisms within its own portfolio and pipeline. Specifically, the CLR Review advocated that in the event of a resumption of policy-based lending, the World Bank should assess the pro-poor nature of public expenditure carefully and ensure that the macrofiscal framework that it supports includes measures to raise Uganda’s low revenue-to–gross domestic product ratio and does not penalize pro-poor spending. In Ukraine, the World Bank program was adjusted to protect poor people after utility tariff increases. But despite appropriate support from investment projects and policy-based operations, targeting accuracy declined because indicators on health and education expenditures were omitted. Similar problems arose in Sri Lanka, where the CPF objective to reduce the prevalence of malnutrition received too little support from the Bank Group to have an impact on the government’s goal of reducing the malnutrition rate and the country program’s objective in this regard was not achieved. These examples show that good intentions must be accompanied by good design of measures intended to protect vulnerable and poor people from negative impacts associated with changing country circumstances or essential reforms.

The Haiti CPF and ISNs before and after the 2010 earthquake are a good example of the Bank Group strengthening its support for the twin goals in a crisis. In the aftermath of the 2010 earthquake, the Bank Group restructured its existing portfolio for immediate relief and recovery efforts. Country circumstances changed so much and the portfolio restructuring was so extensive that it warranted preparation of a new country strategy in the form of two successive ISNs. These ISNs superseded the uncompleted CPF, for which most of the objectives were never achieved. Instead, the ISNs doubled down on reaching poor people and most disaster-affected populations with basic infrastructure and social services, including housing and education.

Realizing the “Knowledge Bank”

The CPF framework is less well suited to clients with more limited demand for IBRD and IDA lending or limited demand for investment lending versus policy-based lending. Resource-rich countries are an example of the former. IEG’s CPEs found that there was no consistent framework for engaging these countries—no common approach to dealing with clients whose defining characteristics were a rich endowment with nonrenewable natural resources and dependence on revenues from their exploitation. In these countries, the challenge for the Bank Group is how to best leverage its knowledge and global experience, and lending is less relevant. EU member states such as Bulgaria and Romania are also examples where demand for lending tends to be limited (except in economic crises), and the Bank Group relies increasingly on reimbursable advisory services (RAS), a form of fee-for-service ASA delinked entirely from lending. Bhutan is an example of a shift away from investment lending toward policy-based lending linked to high volumes of ASA. The team did not adequately adjust the CPF results framework to capture outcomes linked to policy-based lending and dissemination and dialogue on ASA—the only outcomes that could arise from implementation of the Bhutan CPS.

Overall, the Bank Group does not have a standardized M&E system for ASA products. CPF results frameworks tend to be dominated by objectives and indicators related to the lending portfolio, regardless of its relative importance in the mix of interventions in the Bank Group program. Evaluation of CPF programs that tilt heavily toward ASA (including RAS-based programs) are likely to undervalue the influence of the Bank Group program within the country.

Dropped and Canceled Projects—May Be Necessary for Adaptability but Could Involve Possible Trade-Offs with Efficiency

It is expected that some projects planned at the start of the CPF will be dropped or canceled because of changing circumstances, notably a change in government priorities or commitment. Dropped or canceled projects can be a sign of adaptation and innovation. However, in some instances, it is possible that they may represent an inefficiency in the time or money that may have been spent preparing them.

Over the period covered by this RAP, more than one-third of projects (35 percent) have been dropped or canceled (equivalent to 27 percent of the planned lending volume). However, this figure is an underestimate because it applies only to projects that were assigned a project identification number,6 while other projects never reach that stage. The share of dropped and canceled projects has fluctuated modestly across the RAP period, with no clear trend (figure 5.2, panel a).

The reason for projects being dropped is noted in nearly three-quarters of cases (figure 5.2, panel b). The most common reason is that the government has a change in priorities, which may include a central agency (likely finance) blocking the loan from moving forward, even though a line ministry has interest. There is a relatively small, though not a negligible, number of projects for which the World Bank feels that the project no longer fits its strategy.

The possible downsides to dropped projects include the following:

  • Project planning takes time and money for World Bank staff and government representatives. Projects that are planned but are never identified or are canceled or dropped after they are identified may represent an inefficient use of resources.
  • Dropped projects weaken the line of sight from planned Bank Group activities to CPF objectives. The World Bank (2020b) found that more than 80 percent of CPF indicators were dropped or revised at the PLR stage. This high proportion is partly driven by the large number of dropped projects.
  • The analysis of dropped projects in CLRs and CLR Reviews currently does not go much beyond stating the numbers. CLRs and CLR Reviews could usefully examine the reasons for dropped projects and the associated costs in more detail to help inform a discussion on how to ensure that such projects do not represent an inefficiency.

Figure 5.2. Dropped and Canceled Projects

Image
Panel a
A line graph shows modest fluctuations in the share of dropped and canceled World Bank projects during FY13-22.

Panel b
A horizontal bar chart shows that the most common reason for dropped projects is change in government priorities.

Figure 5.2. Dropped and Canceled Projects

Source: World Bank Enterprise Data Catalog.

Note: The drop reason is available for 73 percent of dropped projects. ID = identification number.

Line of Sight

The Bank Group defines the line of sight as “a clear path connecting an activity with its ultimate desired outcome” (World Bank Group 2021c, iv). Within the Bank Group context, the line of sight involves two stages: (i) from Bank Group support to CPF objectives/development outcome; and (ii) from CPF objectives/development outcome to HLOs (defined in box 5.1). The line of sight requires two conditions to be met at both stages: relevance (a necessary condition) and contribution (a sufficient condition).

Regarding the first stage, the RAP found that Bank Group support was not fully relevant to CPF objectives/development outcome. Contribution also had some weaknesses in this first stage in that lending often fell short (that is, planned lending did not materialize and was dropped or canceled, or disbursements were delayed), and adequate restructuring of the country program and the associated results framework was not undertaken even at the PLR stage. Stated CPF objectives/development outcome then tended not to be achieved.

The RAP found that both relevance and contribution were less evident in the second stage, from CPF objectives/development outcome to HLOs, although CPF objectives/development outcome generally mapped well into HLOs. Establishing relevance and contribution tends to be difficult because extraneous influences increase as one moves toward final-stage outcomes, and confounding factors from actions originating outside the Bank Group program come into play.

Table 5.1 summarizes the RAP’s analysis of the line of sight.

Table 5.1. Line of Sight Analysis Based on Completion and Learning Review (CLR) Reviews

Criterion

First Stage

Second Stage

From World Bank Group support to CPF objectives/development outcomes

From CPF objectives/development outcomes to HLOs

Relevance

Weaknesses in relevance

  • There is a lack of selectivity, adaptiveness, and realism in country programs, such as insufficient attention to implementation capacity and political risks.
  • There are weaknesses in CPFs and their results frameworks that insufficiently integrate and leverage ASA and IFC and MIGA support.

Relevance less evident

  • CPF objectives/development outcomes generally map well to HLOs.
  • However, issues remain with the measurement of relevance. Determining Bank Group relevance is complicated by the need to also account for the actions of other development actors.

Source: Independent Evaluation Group.

Note: ASA = advisory services and analytics; CPF = Country Partnership Framework; IFC = International Finance Corporation; HLO = high-level outcome; MIGA = Multilateral Investment Guarantee Agency; PLR = Performance and Learning Review.

Overall, the first stage of the line of sight, where relatively more weaknesses are observed, is under the Bank Group’s control. Bank Group actions that address weaknesses identified in this RAP can help improve both the relevance and contribution of country programs in this first stage.

Improving the relevance and contribution in the second stage is more difficult, given the influence of other development actors, including the government, civil society, and other development partners. For the second stage, further work is needed on approaches to consider the actions of other development actors and on measurement. Regarding relevance in the second stage, it will need to be established up front that every single country program consider the complementary actions of multiple actors that are needed to achieve the HLOs. This is necessary to ensure that the Bank Group is not supporting duplicative or redundant activities or activities that may fail without complementary action. Regarding contribution in the second stage, in-depth, rigorous measurement on a selective case study basis is needed, rather than attempting to measure contribution for every single country program.

  1. For example, IEG’s evaluation of outcome orientation at the country level pointed out that the Bank Group’s “country-level results system does not capture the Bank Group’s contribution to country outcomes well, because it relies on results frameworks premised on metrics, attribution, and time-boundedness that do not fit the nature of country programs well” (World Bank 2020b, x).
  2. During the sample period of FY13–22, the country program was sometimes referred to as the Country Assistance Strategy, Country Partnership Strategy, or CPF. The associated progress reports were referred to as Country Assistance Strategy Completion Report, Country Partnership Strategy Progress Report, and Performance and Learning Review, respectively. These terms are used interchangeably in this RAP.
  3. It is recognized that it may not be possible to discuss some sensitive topics in the CPF document. For such topics, it would still be useful for the Bank Group to develop a more transparent internal process. All other risks can be discussed in the CPF in terms of a change in commitment on key issues such as inclusion or economic reform. The CPF would also identify how the country program would be adjusted if government commitment did not match expectations. The RAP emphasizes that these risks can and should be addressed in the CPF/ Performance and Learning Review. The RAP gives examples of these risks not being well addressed in country programs, suggesting that the issue is not just one of inadequacies in documentation.
  4. It would be useful to examine in the future if weaknesses in risk mitigation explain, for example, the RAP’s finding that when individual CPF objectives/development outcome were considered over the 10-year period of FY13–22, almost half were partially achieved or not achieved. It would also be useful to examine in the future if weaknesses in country-level risk mitigation explain the gap between project ratings and country program ratings in CLR Reviews. The RAP provides examples of country programs where risks could have been better mitigated. With inadequate risk mitigation, future outcome ratings could come under pressure. The RAP also found examples where program changes (possibly in response to risks materializing) were insufficiently documented.
  5. Clearly, the same degree of “oneness” will not be appropriate in every instance—the magnitude and nature of the role that each of the three institutions will need to play will vary from case to case. Senior management signaling and staff incentives are likely to be critical for ensuring well-coordinated, well-complemented, and well-sequenced actions across the three institutions.
  6. A project identification number is generated for a new project once the World Bank team creates the Activity Initiation Summary in the World Bank’s Operations Portal. This happens during the identification and preparation stage and before the Concept Review stage.