Strong leadership at multiple levels in the World Bank enabled innovations and information sharing across sectors, and policy guidance and operational frameworks helped steer the response in the early months.

Coordination of country support was done through portfolio review meetings and typically included the health, education, and social protection sectors. In some countries, coordination might have been improved by including sectors beyond human development.

Crisis instruments, repurposed projects, regional projects, trust funds, and grants, where available, were important for rapid financing in the early days and weeks of the crisis. The Multiphase Programmatic Approach was innovative and useful to quickly expand new lending for the health response.

Operational flexibilities have improved since past crises, but the processing of new projects took several months. Flexibilities in new project processing helped process the Multiphase Programmatic Approach. Countries needed more help with corporate requirements related to gender and citizen engagement.

Procurement was successfully expedited; however, tracking procured goods to ensure that they were received by health facilities was challenged by weak country systems.

Project monitoring systems provided some timely information during the crisis, but integrated country-level monitoring was not always in place.

Existing partnerships involving the World Bank supported preparedness for global and country-level collaboration. Technical partnerships, even though they had limited country coverage, helped expand implementation of interventions.

Pandemic Emergency Financing Facility grants supported COVID-19 plans in collaboration with United Nations partners, although funding was small and not rapid.

The World Bank engaged in close dialogue with partners in the uncertain early months but lacked a financing instrument to help expedite advance market commitments for countries to access vaccines. Multiphase Programmatic Approach vaccine financing was prompt in countries, but supply constraints slowed initial implementation. Stronger regional support in Africa on vaccines earlier in the response may have helped facilitate access to vaccines.

This chapter assesses the quality of the World Bank’s operational policies and partnerships in support of countries during the early response to COVID-19. The assessment is based on dimensions of quality from the theory of action in figure 4.1.

Figure 4.1. Dimensions Assessed for Quality of World Bank Operational Processes and Partnerships to Support Country Responses

Image

Share Tweet Facebook WhatsApp Linkedin Email

Figure 4.1. Dimensions Assessed for Quality of World Bank Operational Processes and Partnerships to Support Country Responses

Source: Independent Evaluation Group portfolio.

Internal Coordination

Strong internal leadership at multiple levels enabled innovations and information sharing that helped coordinate the response. World Bank leadership led a large-scale response supported by a strategy and undertook internal innovations (box 4.1). IDA spending allocations were front-loaded to ensure resources for COVID-19 through an unprecedented move to start Replenishment discussions one year ahead of schedule in April 2021. An emergency operations center led by the Human Development GP and chaired by the Health, Nutrition, and Population GP facilitated technical coordination, adaptive management, and problem-solving for the response. Information sharing among technical staff and across GPs and operational support units increased through, for example, the production of guidance and technical notes, regular learning seminars, and listing answers to frequently asked questions. For example, Social Protection and Jobs; Education; and Health, Nutrition, and Population engaged in the center and developed complementary actions to support the response, building on human capital work done before COVID-19 (World Bank Group 2020a). The Poverty and Equity GP worked with other GPs and the Development Economics Vice Presidency on rapid phone surveys focused on COVID-19. A challenge noted in some cases was aligning actions to draw on technical capabilities of other GPs, such as Water, Transport, Agriculture and Food, and Social Sustainability and Inclusion. These sectors had technical knowledge to support risk communication, vaccine delivery, transport of goods, and sanitation—a first line of defense against COVID-19 before vaccines.

Policy guidelines and frameworks helped guide World Bank teams early in the response. Policy notes developed by GPs and other internal units helped guide actions in the first months of the response. A series of internal learning events for staff also helped. In March 2020, the Health, Nutrition, and Population GP approved its Strategic Preparedness and Response Program MPA, the first phase of which committed $6 billion in funding (World Bank 2020b). The MPA, anchored in a cross-sector response framework, helped guide World Bank teams planning the health and social response in countries. Social Protection and Jobs guided technical planning of World Bank teams by providing a continuously updated database and survey of global social protection responses, including a “Living Paper” detailing the design of support to share country experiences and a social response framework. Country directors and World Bank staff and clients highlighted these social protection resources as being useful in their early policy dialogue to guide response actions. The Education team engaged with partners to develop tools that helped clients implement quality responses, such as modules for phone surveys, COVID-19 impact assessment tools, technical notes for school reopening and teacher performance, and examples of content for remote learning and reports on policy responses (World Bank 2020e, 2021e). The Gender Group, with staff from across GPs, provided a range of resources that defined entry points to address gender in the early response, for example, for vaccination projects. Interviewees reported that these frameworks and tools provided timely and useful guidance to help shape country responses. The early documents and learning developed by GPs and other units also helped shape the Bank Group COVID-19 response framework in June 2020 (World Bank Group 2020b).

The portfolio review process, led by country management, coordinated GP support and was important to identify ways to reorient the portfolio quickly and process requests in the early months of the response. Senior staff in countries noted that in the first months of COVID-19, real-time communication between World Bank country management and global technical teams—especially on the health response—was critical, such as on how to involve key sectors to help identify solutions for reorienting assistance in the country portfolio. In Honduras, project implementation and procurement plans were reviewed through weekly exchanges, and collaboration across GP teams helped reorient the portfolio quickly and process requests from the government. In Madagascar, by repurposing project support, the Urban, Disaster Risk Management, Resilience, and Land; Health, Nutrition, and Population; and Social Protection and Jobs GPs collaborated to facilitate social distancing, hygiene services, and handwashing stations for public transport and to provide cash transfers and cash-for-work activities. India and the Philippines conducted frequent portfolio reviews to coordinate support.

Case studies and the portfolio review suggest that drawing on a wider range of World Bank sectors helped the response. Forty percent of projects collaborated with one other GP (figure 4.2). Collaboration varied by instrument—development policy financing (DPF) and catastrophe deferred drawdown option (CAT DDO) had the highest collaboration, as did CERC and regional projects, although to a lesser extent (more than half of these projects had GP collaboration). About 60 percent of World Bank country programs had collaboration involving at least one sector beyond the Human Development GPs, such as Finance, Competitiveness, and Innovation; Agriculture and Food; Social Sustainability and Inclusion; Water; and Governance. More frequent collaboration with a range of GPs would have helped support local government, sanitation, gender, psychosocial support, nutrition, citizen engagement, and other areas. One way that collaboration was encouraged was through joint projects that engaged Health, Nutrition, and Population, but it was also important to align support across GPs, such as to the MPA, to help address interrelated needs. Examples of where Health, Nutrition, and Population collaborated with other GPs to draw on a wider range of support include the following:

• In Tajikistan, the MPA led by Health, Nutrition, and Population collaborated with Social Protection and Jobs to implement social protection support that included an aligned ASA led by the Governance GP to assist third-party monitoring of the health response.
• In the Philippines, the MPA led by Health, Nutrition, and Population collaborated with Digital Development to digitalize systems and with Social Sustainability and Inclusion to support stakeholder consultations.
• In West Africa, Health, Nutrition, and Population and Agriculture collaborated in regional project support to train One Health agents in community-based surveillance.
• In Uganda, the DPF led by Macroeconomics, Trade, and Investment provided support across sectors to ensure basic utilities and water services, expand social registration, provide vouchers for farmers, enact a child protection policy, and procure medical supplies, in collaboration with Health, Nutrition, and Population; Agriculture and Food; Social Protection and Jobs; Social Sustainability and Inclusion; Water; Energy and Extractives; and Governance.

Figure 4.2. Global Practices Contributing to Projects for Early Health and Social Response

Image

Share Tweet Facebook WhatsApp Linkedin Email

Figure 4.2. Global Practices Contributing to Projects for Early Health and Social Response

Source: Independent Evaluation Group portfolio.

Note: Percentages can add to more than 100 percent across contributing Global Practices or Global Themes because a single project may have multiple contributing Global Practices or Global Themes. The analysis looks at collaboration in parent and additional financing projects. Emergency instruments include COVID-19–activated Contingency Emergency Response Component and catastrophe deferred drawdown option projects. N = 253 parent and 67 additional financing projects. DPL = development policy loan; IPF = investment project financing; MPA = Multiphase Programmatic Approach; PforR = Program-for-Results.

Instruments Supporting COVID-19 Crisis Response, Streamlined Processes, Corporate Requirements, and Procurement

The MPA provided an innovative and rapid approach to expand new lending for the health response. The first MPA, approved in April 2020, included project financing in 51 countries, mainly investment project financing (IPF). The additional financing approved in October 2020 expanded support to 70 countries by April 2021. The MPA offered an umbrella approach with a menu of areas that eligible countries could adapt to their needs and build on to address subsequent stages of the response. This allowed projects to maintain some uniformity in content, which increased the speed of design, processing, and approval while adapting content to country demands. The first MPA projects in countries disbursed in April 2020, then disbursement increased gradually throughout the first 15 months of the response as more projects became active. By the end of May 2020, three months after the crisis was declared, all approved MPA projects had made initial disbursements, with subsequent disbursement by August 2020 (appendix H analyzes the early support of the health MPA).

Complementing new project support with advisory services, the use of DPFs and immediate release of financing in the early days and weeks of the response was important to support quick actions. Ministries of health were overwhelmed and often needed immediate financing and health advisory and technical support to respond to COVID-19. In some countries, such as India, Mozambique, and Senegal, crisis and repurposed resources were used for the immediate response, which provided an opportunity for staff to support the early health dialogue. Drawing on crisis instruments, regional projects, ASA, and repurposed projects, where available, ensured quick financing by March 2020, which new project support could build on later. The minimum time for countries to process and disburse MPA financing was about two months. New DPFs first disbursed in May 2020 and were useful for rapid expansion of crisis support for vulnerable groups, surveillance systems, or new policies to protect child welfare, but they were limited in coverage. New IPF projects to support social protection and education were processed throughout the first year of the response, with the first disbursement in July 2020. The portfolio analysis shows the mix of instruments used across countries to support the response (figure 4.3). Findings from the portfolio review, case studies, regional project analysis, and operational process review (appendixes B, C, and G, respectively) point to the use of instruments for supporting different time frames and aspects of crisis response (box 4.2).

Figure 4.3. Mix of Instruments Used by Global Practices to Support the Response

Image

Share Tweet Facebook WhatsApp Linkedin Email

Figure 4.3. Mix of Instruments Used by Global Practices to Support the Response

Source: Independent Evaluation Group portfolio.

Note: The analysis is based on 253 projects coded for the evaluation. COVID-19–activated CERC includes 26 projects in eligible countries and selected lead Global Practices with COVID-19 emergency response tags or keywords “COVID” or “corona” in their titles, project development objectives, indicator, or summary text. This excludes 13 COVID-19–activated CERC projects: 10 projects reported in the Global Facility for Disaster Reduction and Recovery CERC dashboards (May 5, 2021, and June 1, 2021) in eligible countries and lead Global Practices that did not have COVID-19 emergency tags or keywords at the date of data extraction (May 12, 2021), and 3 projects identified separately by the Independent Evaluation Group later in the evaluation. CAT DDO = catastrophe deferred drawdown option; CERC = Contingency Emergency Response Component; DPL = development policy loan; IPF = investment project financing; MPA = Multiphase Programmatic Approach; PforR = Program-for-Results.

Crisis instruments helped World Bank country programs to provide resources to governments rapidly and flexibly in the early months of the COVID-19 response, yet few countries were prepared with good coverage of these instruments in their portfolios. About 65 percent of countries had CERCs in their portfolios in the 15 months leading up to COVID-19. Twenty two percent had a high proportion of CERCs in their portfolios (figure 4.4), and about 6 percent had CAT DDO to support the early health and social response. Case studies found that multiple CERCs in a variety of sector projects provided flexibility to manage resources across the portfolio, but few World Bank country programs had planned CERCs in this way. Moreover, innovating on the design of CERCs—for example, to support local government responses—could improve their use in crises. In the Philippines, an innovative community-based disaster response modality supported by the World Bank provided resources for subnational COVID-19 responses. Challenges in using CERCs were related to their activation timeline, which required countries to declare an emergency to make resources available. Case studies found that this meant it was not possible to access funds for planning activities before the crisis struck—adjusting this requirement could be important to improve the flexibility of CERCs to provide immediate resources for prevention activities to avert a future crisis. There was also concern about crisis instruments diverting financing from projects, without commitments to replenish those resources. However, in countries with experience using crisis instruments, CERCs provided rapid financing, and there was good preparedness with multiple CERCs in the portfolio:

• In India and Mozambique, multiple CERCs allowed for flexible management of resources across sectors. Previous crisis experience in Mozambique led to careful planning of CERCs before COVID-19, ensuring availability of resources from these and from repurposed projects for immediate response and creating space to develop an MPA project by June 2021 focused on vaccines and continuity of health services. Where available, countries often used CERCs to procure medical supplies at the onset of the crisis.
• In Honduras, the adjusted country portfolio drew on CERCs and a CAT DDO, which supported collaboration across GPs to tackle the impact of hurricanes in addition to COVID-19. The CAT DDO included prior actions to help the national and subnational emergency response plans and to strengthen health surveillance systems.

About 70 percent of countries successfully received disbursements from COVID-19 projects in the first three months of the response, starting immediately when the crisis was announced in March 2020—a rapid pace for the World Bank. Key was having some early financing from crisis instruments, repurposed projects, regional projects, trust funds, or grants resources within days. The fastest disbursements in the first days and weeks, before other project financing started, were from crisis instruments, repurposed projects, and regional projects, pointing to their role in swift early disbursement in a crisis (figure 4.5, panels a and b). MPA and DPF financing started to disburse several months into the crisis. New IPF and Program-for-Results slowly increased their share of disbursements, indicating their ongoing role in supporting countries for recovery, along with later phases of the MPA. Having flexible World Bank–executed trust fund resources was key to provide just-in-time assistance to countries, such as to plan activities at the start of the crisis and even before it was officially announced. Case studies also noted the importance of having grant resources for quickly processing new projects in some countries where government was hesitant to use scarce IDA resources for the crisis. In some countries, it took parliament about a year to approve new emergency projects, limiting their value for the emergency response, whereas grant financing could be approved rapidly.

Figure 4.4. Crisis Instruments in Country Portfolios

Image

Share Tweet Facebook WhatsApp Linkedin Email

Figure 4.4. Crisis Instruments in Country Portfolios

Source: Independent Evaluation Group portfolio.

Note: The figure is based on a combined 167 parent and additional financing projects from eligible countries tagged with CERC (152 projects) or a CAT DDO (15 projects), irrespective of Global Practice and active between February 1, 2020, and April 30, 2021. The figure includes an estimated COVID-19–activated CERC of 55 projects across Global Practices: 47 projects from the Global Facility for Disaster Reduction and Recovery CERC dashboards updated on May 5, 2021, and June 1, 2021, plus an additional 8 projects identified by the Independent Evaluation Group portfolio analysis. Regional projects with CERC are excluded. The level of emergency instruments is based on the quantity of overall CERC and CAT DDO projects per country, with levels broken down by tercile. Low level of emergency instruments: 0 to 1 CERC or CAT DDO project; medium level of emergency instruments: 2 CERC or CAT DDO projects; high level of emergency instruments: 3 to 10 CERC or CAT DDO projects. The total number of countries is 98. CAT DDO = catastrophe deferred drawdown option; CERC = Contingency Emergency Response Component.

Having a mix of instruments in the portfolio that could be used at different times frames in the early weeks and later months of the crisis response was important. Table 4.1 synthesizes findings on various instruments used in the COVID-19 response in terms of their timing, constraints, and opportunities. Countries with regional projects, better embedding of crisis instruments in the portfolio, experience using a range of instruments to support human capital, and trust funds were often better prepared for the crisis response.

Figure 4.5. Disbursement of COVID-19 Resources by Instrument and Time

Image

Share Tweet Facebook WhatsApp Linkedin Email

Figure 4.5. Disbursement of COVID-19 Resources by Instrument and Time

Source: Independent Evaluation Group portfolio.

Note: Disbursement data are retrieved from the World Bank’s Standard Reports. In panel a, disbursements reported are adjusted using the share of COVID-19 response content estimated in the coding of each project by the Independent Evaluation Group and combines parent projects and additional financing. New projects (approved on or after February 1, 2020) are assumed to have a 100 percent share of COVID-19 response content. Refer to appendix B for portfolio description. Panel b is organized from fastest to slowest time to first disbursement from the start of the crisis. The time to first disbursement is defined as the number of months between February 1, 2020, and the first disbursement date during the evaluation period. CAT DDO = catastrophe deferred drawdown option; CERC = Contingency Emergency Response Component; DPL = development policy loan; IPF = investment project financing; MPA = Multiphase Programmatic Approach; PforR = Program-for-Results. N = 246 projects.

Sources: International Evaluation Group portfolio; World Bank 2012, 2017.

Note: The evaluation did not assess the extent of use of trust funds. CAT DDO = catastrophe deferred drawdown option; CERC = Contingency Emergency Response Component; PforR = Program-for-Results.

Streamlined Operational Processes

Operational flexibility facilitated rapid processing of new projects for the health response. All projects followed processing guidelines for emergency situations, which bypasses the concept stage—this flexibility was in place before COVID-19. Guidance to process new projects was provided by the fast-track facility. GPs set up technical committees for quick peer review of projects. Additional flexibilities that helped process the MPA quickly were shortened clearance deadlines and delegated approvals to speed up project processing, paused gender tagging for the first MPA, waivers for financing food expenditures under the IPF policy, and encouragement to use project preparation advances without submitting individual requests. These flexibilities helped Health, Nutrition, and Population process new projects for the response, but these flexibilities did not apply to other GPs, and additional steps were introduced for some new projects. For example, new two-page justifications were sometimes required of staff before regional operations committee meetings for projects viewed as risky or large. Given the emergency and its extensive impact, flexibilities applied to the MPA could have been applied to all new financing to help GPs support countries quickly. This was especially important, given the limited use of additional financing compared with past crises. A previous evaluation reported the median time from approval to first disbursement as 4.8 months for new crisis projects, 7.1 months for noncrisis projects, and 1.5 months for DPFs (World Bank 2019a). MPA projects and new DPFs for COVID-19 were very quick to disburse, compared with past crises—the median time from approval to first disbursement for MPA projects was 1.5 months, and the median time was less than 1 month for DPFs. The median time was about 5 months for new IPFs, similar to past crises. However, some projects disbursed in less than 1 month from approval.

Compared with past crises, the World Bank’s operational agility improved in COVID-19, but challenges remained. Implementation Status and Results Reports identified new project processing as a challenge in COVID-19, though less so than in past crisis (figure 4.6; appendix F). There was strong hand-holding support to staff, but templates, technical specifications, and guidance on safeguards and procurement were often developed in real time, resulting in confusion. Templates and guidance were often developed several months into the response and then revised, requiring countries to retrofit project information to the templates. Once guidance stabilized, and there was agreement on how to proceed, project teams often reported easier processes. Teams that processed MPA projects later in the response reported smoother processes because problems had been resolved. Interviewees noted the need to review emergency processes to eliminate confusion on guidance in future crisis responses. A previous evaluation identified the importance of a road map for crisis engagement that defines, for example, broad divisions on roles and responsibilities and the rationale, modalities, and instruments in responses (World Bank 2012). Furthermore, the Organisation for Economic Co-operation and Development emphasizes that pandemic preparedness requires detailed and up-to-date operational plans and processes describing the different roles of staff, procedures, and uses of instruments in responding to crises (OECD 2022).

Figure 4.6. Operational Process–Related Success and Challenges Reported by Projects

Image

Share Tweet Facebook WhatsApp Linkedin Email

Figure 4.6. Operational Process–Related Success and Challenges Reported by Projects

Source: Independent Evaluation Group review of project Implementation Status and Results Reports for lesson analysis.

Note: The COVID-19 portfolio includes 120 successes and 165 challenges in the selected areas of the figure, extracted from 113 projects coded with those areas. “Partnership and collaboration” looks at partnerships to support implementation, information sharing, and joint analyses. “Flexibility in operational procedures” looks at the timely processing of new financing for the response. “Streamlining implementation support to projects” looks at factors that facilitated clients to rapidly implement support in projects (appendix F).

World Bank staff responded to enormous demands to deliver extraordinary support in unprecedented circumstances. Staff worked for long hours to deliver new and repurposed operations, without travel, while learning to use remote connectivity and geospatial tools and adapting to home-based work amid evolving family arrangements and stresses. Some staff suffered personal losses; many coped with pandemic-related anxiety. Heavy work pressures stemmed not only from mounting the COVID-19 response but also from other ongoing considerations: the locust response, natural disasters, debt relief efforts, the July 2020 staff realignment, other internal reorganizations (such as splitting the Africa Region into two Regions), and a leadership rotation in the Human Development Practice Group.

Access to surge capacity to process early health support was critical, but internal mobilization of this capacity was challenging. As the crisis continued, managers in health mobilized surge capacity—using retirees, exchanging overloaded staff, and increasing the responsibilities of country office staff—to help quickly process projects. The operational process review suggests that further surge capacity to process early health support could have come from within the World Bank. Social Protection and Jobs; Education; and Health, Nutrition, and Population were processing 240 percent to 480 percent more projects, whereas other GPs were processing fewer projects than before. The operational process review findings also point to a challenge in the incentives to collaborate across GPs at the operational level to help process and implement emergency project support, such as a DPF or MPA led by one sector, as each GP has its unique sector focus.

Gender, Citizen Engagement, and Safeguards

GPs addressed gender requirements more consistently when support predating COVID-19 could be adjusted. Addressing gender requirements with new support in a crisis was challenging for World Bank teams without technical assistance to help consistently incorporate gender approaches in the response. In Social Protection and Jobs, earlier analysis of gender equality was applied in COVID-19 projects, including the expansion of registries with an emphasis on female-headed households and women with young children. Opportunities to address gender in the health response were missed, and gender analysis in the Health, Nutrition, and Population response is rare. An internal review by the Gender Group of 27 MPA projects highlighted 3 projects that intentionally addressed gender equality (about 10 percent): projects in Pakistan and Sierra Leone defined gender activities as part of project components, and the Mongolia project included a section on gender differentials. Project teams noted a need for more technical support to implement well-developed gender approaches in health and education projects. In Uganda, Social Sustainability and Inclusion reviewed COVID-19 projects to identify opportunities to address gender, which helped GPs to plan support.

Outside specific innovations, citizen engagement received limited emphasis across GPs. Some interviewees noted that the current approach to citizen engagement had limited applicability in a crisis and often was not developing country systems that could be drawn on during COVID-19. For example, the citizen engagement intranet site has very few updates and little content on COVID-19. Helpful innovations used by health and education projects included SMS feedback from citizens about services. Having platforms fostering citizen engagement in place before the next crisis could be important because lack of trust and accountability and the weak roles of citizens were challenges to mobilizing demand for services during the COVID-19 response.

The new Environmental and Social Framework was challenging amid the early months of the crisis. Extensive support was provided to World Bank teams to implement the Environmental and Social Framework, including help with completing documents. COVID-19 guidance released early in the pandemic (in response to operational realities) allowed for the Environmental and Social Framework requirements to be met after the project was approved, although they needed to be completed before processing procurement of medical goods. Despite this much appreciated support, because the Environmental and Social Framework was new, interviews noted that developing the required outputs and processes involved on-the-job learning, which was challenging because clients—especially in health for processing the MPA—were overwhelmed with urgent COVID-19 demands. The outputs required from clients included the Environmental and Social Review Summary, Environmental and Social Commitment Plan, Stakeholder Engagement Plan, and documents for risk-related topics, such as medical waste, social inclusion, and nondiscrimination. The World Bank used additional financing in other crises to provide flexible and fast-disbursing support, which built on existing safeguards to avoid adding new steps in a crisis. However, the use of additional financing was initially limited in COVID-19 operations because projects approved under the previous safeguards policies were required to transition to the new Environmental and Social Framework until a waiver was issued in June 2020. More streamlining of safeguard requirements for new projects in the early crisis response could have been helpful.

Procurement of Goods

The relief stage supported emergency procurement of medical and other goods, with successful efforts to expedite processes. By June 2020, five months into the response, almost 50 percent of procurement contracts supporting the early COVID-19 response were signed, ensuring that countries received protective gear and medical supplies and equipment. In alignment with the World Bank’s Guidance on Procurement in Situations of Urgent Need of Assistance or Capacity Constraints, procurement was supported through countries adopting accelerated procedures, World Bank–facilitated procurement, and collaboration with United Nations organizations. Accelerated processes included allowing for retroactive financing, direct selection, electronic bidding by email, and larger up-front payments to secure goods on global markets. Although steps were not always clear, and case studies noted an opportunity to clarify guidelines on accelerated procurement procedures, procurement in COVID-19 was quick and smooth compared with past crises, helping to obtain needed medical supplies. A previous IEG evaluation of support to natural disasters identified procurement as a main constraint (World Bank 2006), whereas in COVID-19, procurement was not a main challenge reported in Implementation Status and Results Reports or interviews, despite global supply chain difficulties.

In COVID-19, countries successfully employed a variety of mechanisms to procure needed goods quickly, including procurement using United Nations agencies and World Bank–facilitated procurement. This may have been difficult before the World Bank’s procurement reform in 2016. Close support from World Bank procurement specialists was key, and in some cases, senior procurement staff tracked health goods to ensure that they arrived rapidly in the intended country. Direct selection was used for almost 60 percent of contracts. There was World Bank–facilitated procurement of scarce medical goods. Some World Bank teams preferred to use United Nations agencies for procurement, given their simplified processes, but countries that used World Bank–facilitated procurement reported that the service helped reduce the cost and speed of receiving goods where it was used. In total, the World Bank procured just under $1.9 billion of critical goods in the early response, including COVID-19 test kits, personal protective equipment, facility improvements, laboratory and medical equipment, and technology (figure 4.7). About 67 percent of the procurement of goods for the health and social response was supported by the MPA, making it a critical vehicle for emergency financing.

Figure 4.7. Cumulative Procurements for Goods by Category, by Date

Image

Share Tweet Facebook WhatsApp Linkedin Email

Figure 4.7. Cumulative Procurements for Goods by Category, by Date

Source: Independent Evaluation Group portfolio.

Note: The data include signed procurements only, which means active contracts with a signature date between February 1, 2020, and April 30, 2021, and a status of signed, under implementation, completed, or under review. Signed procurements also include active contracts with a signature date after February 1, 2020, no contract sign date, and a status of signed or under implementation. For procurements without a contract sign date, the sign revision date was used. The number of procurements per category was calculated by separating 2,690 individual goods procurements into components against a defined taxonomy using text analytics. The number of procurements of each good is 8,565.

Key challenges to procurement included the coordinated planning and tracking of goods until their arrival in health facilities, and limited emergency preparedness of procurement systems in countries. World Bank teams worked closely with partners to plan what was procured, but it was challenging, and in some cases, goods were already procured by another partner by the time they arrived, and another item was needed more urgently. In some countries, World Bank teams helped develop systems to plan and track goods until the arrival at health facilities, which helped monitor what was purchased by different partners and ensured accountability of goods arriving to benefit communities. In Paraguay, the health ministry used georeferenced data to plan procurement. Tajikistan and Zimbabwe developed electronic tracking systems, including an SMS platform to verify receipt of goods. In these countries, health sector supply chains were strengthened in real time. World Bank teams also worked with the government to adopt accelerated procurement processes so they could procure goods more quickly using country systems. Such support was provided in Djibouti and by the Africa CDC and Organisation of Eastern Caribbean States regional projects.

Monitoring and Reporting

World Bank projects planned indicators, although timely monitoring was challenging. The review of indicators shows that countries planned indicators to measure 60 percent of response areas (figure 4.8). Few indicators measured child welfare and social services, and measurement of community engagement was even more limited. Vaccination is measured less, mainly because vaccine projects were developed later in the response. However, less than 40 percent of indicators were tracked during the response, given that project reporting is every six months. A review on the first year of the pandemic highlights that nearly every report identifies the lack of data collection as an important constraint in understanding funding, activities, and results (Johnson and Kennedy-Chouane 2021). Monitoring of indicators from new projects was especially challenging; however, the health response stands out as having good monitoring, compared with other areas because Health, Nutrition, and Population monitored the new MPA projects closely (figure 4.9).

Figure 4.8. Country-Level Alignment of Indicators to Measure the COVID-19 Response

Image

Share Tweet Facebook WhatsApp Linkedin Email

Figure 4.8. Country-Level Alignment of Indicators to Measure the COVID-19 Response

Source: Independent Evaluation Group portfolio.

Note: The figure shows the proportion of countries with at least one indicator in a project in their portfolio to monitor a planned COVID-19 thematic response area. “Ensure health services” includes indicators measuring achievements of health support in the COVID-19 response. “Protect the poor and vulnerable” includes indicator measuring social protection and informal sector improvements. “Ensure child welfare and social services” includes indicators measuring education, nutrition, and psychosocial improvements. “Community engagement” includes indicators measuring aspects of social cohesion and citizen engagement. “Institutional strengthening of response” includes indicators measuring system and policy improvements. The total number of indicators is 2,219 and countries is 92.

Indicators frequently measured outputs, but measurement of the quality of services, social cohesion, and systems improvement was limited. Indicators often measured materials received, such as equipment, and in some cases, the delivery of the service. These data will need to be complemented by data collection on quality improvements and outcomes to assess the achievements of support to restructure systems (box 4.3). Moreover, there is limited measurement of and support to community activities, especially social cohesion.

Figure 4.9. Extent of Monitoring in Countries by Response Area

Image

Share Tweet Facebook WhatsApp Linkedin Email

Figure 4.9. Extent of Monitoring in Countries by Response Area

Source: Independent Evaluation Group portfolio.

Note: The figure reports the share of countries within each lending group and response area by level of monitoring. Level of monitoring is based on the tracking or progress share of indicators in a country response area: low (0 percent to 22 percent); medium (22 percent to 52 percent); high (52 percent to 100 percent). IBRD and blend: n = 39 countries; IDA: n = 49 countries. N = 88 countries. Regional projects are excluded from the analysis. IBRD = International Bank for Reconstruction and Development; IDA = International Development Association.

About two-thirds of countries show evidence of positive progress on tracked indicators. Early progress is most notable in IDA countries (figure 4.10), where a medium to high share of tracked indicators show early progress, especially for Africa. A little more than 40 percent of all indicators assessing the health response (ensuring health, risk communication, vaccination, and strengthening of health systems and coordination) show early progress. There is less progress for indicators measuring results in social protection and education, with only about one-quarter showing evidence of progress.

Figure 4.10. Countries by Extent of Level of Progress on COVID-19 Response Indicators

Image

Share Tweet Facebook WhatsApp Linkedin Email

Figure 4.10. Countries by Extent of Level of Progress on COVID-19 Response Indicators

Source: Independent Evaluation Group portfolio.

Note: The figure shows the percentage of countries by levels of evidence of progress of indicators, where levels correspond to terciles of the distribution of the evidence of progress share of total indicators across countries: low (0 percent to 16.67 percent), medium (16.67 percent to 51.76 percent), and high (51.76 percent to 100 percent). Indicators were coded based on whether the monitoring value showed no change or adverse change from the baseline or evidence of progress toward the target. The total number of countries is 88. IBRD = International Bank for Reconstruction and Development; IDA = International Development Association.

Integrated reporting of data on various areas of the World Bank’s COVID-19 response was important to discussions with the country government, World Bank teams, partners, and headquarters. World Bank country management noted that it was challenging to compile data across projects to get a full picture of progress on the response for weekly or monthly meetings. Staff faced many fragmented requests from headquarters, GPs, and others to report on projects. The challenge of monitoring during the crisis reinforces the value of investing in better routine data systems for adaptive implementation of country support (discussed in chapter 3), such as geo-enabled monitoring, SMS, and dashboards, which were used in some countries in Africa and Latin America. The operational process review suggests that improving integrated reporting could benefit World Bank portfolio monitoring and policy dialogue during a crisis but also more routinely. The systems set up for monitoring the vaccine response provide an example of a coordinated approach to country program monitoring that aligns with partners and strengthens country systems (Chazaly and Goldman 2021). Box 4.4 provides examples of integrated reporting by World Bank GPs.

Partnerships to Facilitate Response

Partnerships between development organizations and the World Bank helped preparedness and crisis responses in countries. Partnerships developed by GPs and country management units before COVID-19 often provided technical knowledge and financing for the health and social response. Having such relationships defined before COVID-19 helped the partners to align strategies to rapidly collaborate on COVID-19 support in countries, as other evaluations have also found (Johnson and Kennedy-Chouane 2021). Important partnerships identified in interviews, documents, and case studies included development organizations such as United Nations’ agencies, multilateral development banks, bilateral donors, foundations, nongovernmental organizations, and global partnership organizations (Global Partnership for Education [GPE]; Global Financing Facility; Education Technology; Gavi, the Vaccine Alliance; and the Global Partnership for Social Accountability). The World Bank engaged in four main types of partnerships to support the COVID-19 response:

• Financing partnerships to provide grants within World Bank projects or aligned complementary financing support to the World Bank’s response in countries
• Technical partnership activities to innovate and expand knowledge for quality implementation of response activities, often supporting analytic work to understand the status of the response
• Procurement and supply chain partnerships to improve the purchase and delivery of goods
• Nongovernmental partnerships to support provision and feedback on services at the front line of the response

At the global level, partnerships also provided guidance on emergency plans and quality standards for the health response in countries. Key global partnerships are described in box 4.5.

Global partnerships in health and education were quickly adapted to support the COVID-19 response in countries. In education, working with the GPE partnership consortium (which includes World Bank, UNICEF, and the United Nations Educational, Scientific, and Cultural Organization), partners coordinated technical and financing support rapidly to support the response with wide coverage of countries. Case studies highlighted the success of GPE grants to quickly expand education support, collaborating with UNICEF and other partners. In Rwanda, GPE supported approaches to remote learning, including broadcasting lessons on the radio and free SMS messages. The Education Technology team facilitated rapid sharing of technical knowledge with partners and countries to expand technology-based learning. For example, more than 30 continuity stories from around the world highlighting countries’ remote learning solutions were developed in partnerships with the Organisation for Economic Co-operation and Development, Harvard Global Education Initiative, and HundrED. The approach it employed is a model that could be expanded to other sectors. In health, the Global Financing Facility provided just-in-time knowledge support for health services and strategies. However, countries could have benefited from earlier technical knowledge to continue essential health services and for gender and risk communication. The World Bank’s partnership with Gavi, the Vaccine Alliance was key in countries such as Tajikistan, but more countries could have benefited from this type of early support on vaccines during COVID-19—a theme that also arose in IEG’s Global Program Review (World Bank 2014). Global Partnership for Social Accountability support in citizen engagement stands out for addressing a key need, but the limited coverage of countries was a challenge to strengthening accountability mechanisms, trust, and participation. Procurement partnerships were helpful across sectors. Health guidance partnerships with WHO were important for aligning health actions in countries; nevertheless, challenges arose because of the emphasis of this guidance on the health sector and less so on the social impacts of COVID-19 (discussed in chapter 2) and of alignment on vaccines (discussed under vaccine support).

Financing and technical partnerships helped expand response actions quickly in countries, although technical partnerships were rare. Funding partnerships were common for coordinating support. In India, engagement with the Asian Development Bank provided an additional $500 million that followed prior actions similar to those of the World Bank’s social protection and education support. Also in India, financing for the World Bank’s health project was coordinated with the Asian Infrastructure Investment Bank. In Honduras, the World Bank collaborated with the Inter-American Development Bank in financing the social response, including joint missions. In the Philippines, the Asian Development Bank and the World Bank also complemented each other by jointly financing civil works for laboratory structures and equipping laboratories. Technical partnerships were important for expanding COVID-19 support, although rare in countries, especially in the health sector. For example, the World Bank and UNICEF jointly expanded remote learning and psychosocial services in Djibouti, and the World Bank helped the government expand its vouchers and school feeding interventions with the World Food Programme.

Collaboration with nongovernmental organizations and the private sector, even though limited, helped expand community-based implementation and use of technology. For example, in Tajikistan, the World Bank worked with the Open Society Assistance Foundation to support civil society capacities to prevent and respond to COVID-19 and with the Aga Khan Development Network to train health workers. In Togo, the new Novissi social protection program helps citizens who lost their income and were pushed into poverty by the pandemic. Supported by the World Bank to undertake phone surveys, the government partnered with the University of California and the nonprofit GiveDirectly to prioritize those most in need by using satellite imagery, mobile phone data, and nationally representative household consumption data (World Bank 2022a). In Belize, the World Bank project partners with the national bank and telecommunications company to deliver cash transfers. Beneficiaries are notified of the funds by SMS message, and they collect their payment through a network of agents. A few examples of partnerships with nongovernmental organizations also occurred. Further collaboration with nongovernmental organizations could have helped expand technical learning and community activities.

Case studies noted a challenge in providing technical assistance to help countries expand quality critical health services. Partnerships in this area were limited. Countries required technical learning to implement new surveillance and case management approaches. Where World Bank teams already had technical knowledge implementing critical health services, they could apply that to COVID-19. Some countries benefited from technical knowledge partnerships through projects before COVID-19. For example, the 18th Replenishment of IDA (IDA18) and the 19th Replenishment of IDA committed to support pandemic preparedness plans in 25 countries (IDA 2017, 2020). During COVID-19, the Regional Disease Surveillance Systems Enhancement Project in West Africa coordinated technical learning through its cross-country exchanges, including with the Africa CDC, UNICEF, and WHO. Similarly, the Organisation of Eastern Caribbean States Regional Health Project supported technical knowledge learning with the Pan American Health Organization and the US Centers for Disease Control and Prevention. Technical learning partnerships to improve preparedness may be scaled up to other countries through such regional collaboration.

Pandemic Emergency Financing Facility

In coordination with partners, PEF grants provided support for COVID-19 plans; however, the amounts were small, and the funding did not support just-in-time actions in the first weeks of the response. The PEF was set up in 2016 through partnership and working with the private sector, with funding provided by Australia, Germany, IDA, and Japan and also insurance coverage through World Bank catastrophe bonds and insurance-linked swaps (World Bank 2020c). PEF provided surge financing to IDA countries facing large cross-border disease outbreaks, such as Ebola, and to catalyze the creation of a global market for pandemic insurance instruments. The WHO pandemic declaration triggered the PEF’s insurance window, and allocations were defined in a little more than a month. All $196 million of the fund was transferred to support COVID-19 responses in 64 countries by September 2020 (World Bank 2020c). The timeliness of PEF resources for just-in-time use by World Bank teams was limited by the need to declare an emergency to access the funding and by the processing requirement that PEF had to be included in a World Bank financing project for recipient execution. The amount of PEF financing might have been more useful with payment divided among fewer countries. Alternatively, PEF might have been more useful and timely had it been provided to health teams as a World Bank–executed trust fund for just-in-time financing of joint ASA to inform country and partners responses. Many governments struggled with how to respond to COVID-19 and sought just-in-time ASA in health, especially diagnostics and technical assistance, to refine strategies and planned actions. Such support was critical throughout the early response for adaptive management but especially so in the early weeks and months of the pandemic. Flexibility to use PEF in this way could have increased financing for just-in-time ASA by about threefold, given the limited allocation of about $60 million to ASA.

The flexibility of PEF to finance United Nations partners to facilitate coordination in countries and the quick processing of its grants were key assets. More than 60 percent of PEF grants were used to finance United Nations partners; the rest were processed in World Bank projects. In Honduras, the Pan American Health Organization received World Bank PEF support to adapt and equip seven health establishments for COVID-19 between July 2020 and March 2021. In Mozambique, the United Nations Population Fund received PEF resources to continue support to sexual and reproductive health services during COVID-19. In Uganda, PEF was integrated in MPA financing, allowing for faster processing over IDA credit resources, and the funding was used to procure health-related goods, training, and other support for the national COVID-19 plan.

Vaccine Support

In a context of high uncertainty in the early months, the World Bank engaged with global partners in efforts related to vaccines but lacked an instrument to help expedite country vaccine access. In the first months of COVID-19, Health, Nutrition, and Population convened with global partners to explore ways to help low-income countries to access vaccines when they became available on global markets (see table 4.2 for a timeline of vaccine response). In addition to competing with wealthier countries, securing advance access to vaccines carried a significant risk because it was not known if vaccines would be successful—IDA countries could not assume this burden of risk without global assistance. Earlier, for the pneumococcal vaccine, the World Bank had gathered grant resources together to aggregate demand for vaccines across countries (Cernuschi et al. 2011). However, a World Bank instrument was not available for quick use in a crisis context to aggregate demand across countries for advance market commitments other than convening actors for grant resources. Another approach could have been to secure earlier commitments for donations from high-income countries, as advocated outside the evaluation period by the Multilateral Leaders Task Force on COVID-19 in June 2021. Organizing these actions would have required strong commitments from high-income countries to act quickly early in the response.

The World Bank’s early convening with partners helped form the Access to COVID-19 Tools Accelerator partnership and coordinate support to vaccines and health systems at the country level. In June 2020, the World Bank committed to co-convene the Access to COVID-19 Tools Accelerator health systems and vaccine pillars (the vaccine pillar became Country Access to COVID-19 Vaccines [COVAX]), but in July 2020, the World Bank decided not to co-convene COVAX and focused on country-level support through the health systems pillar. The health systems pillar could be supported through the MPA and existing country relationships. The World Bank remained involved in COVAX but not as a co-convener, and it focused on the country level, where it had good access to financing and country relationships to support response actions. This decision was taken within a challenging global context marked by much uncertainty about vaccines and extensive internal discussion about how to support advance market financing for vaccines without an appropriate global-level instrument. At the country level, the focus was on the MPA financing for vaccines and engagement in country-level committees to plan for vaccines. Having a global-level instrument that allowed for advance market commitment could have helped increase the value added of the COVAX partnership to help low-income countries access vaccines one year earlier. Stronger global-level engagement of World Bank expertise in COVAX early on, as was seen later in response, may have helped facilitate earlier access to global supplies, manufacturing, and risk pooling arrangements for IDA countries.

Sources: Independent Evaluation Group portfolio; Dalberg Advisors 2021; Van Trotsenburg 2021; WHO 2021c; World Bank 2021a, 2021b, 2021h.

Note: The COVAX partnership within ACT Accelerator is for COVID-19 vaccine development, manufacturing, and equitable access for countries. ACT = Access to COVID-19 Tools; Africa CDC = Africa Centres for Disease Control and Prevention; COVAX = Country Access to COVID-19 Vaccines; UNICEF = United Nations Children’s Fund; WHO = World Health Organization.

Engagement of regional organizations in COVID-19 vaccines was important for coverage in Africa. The World Bank also engaged in early discussions in 2020 with the Africa CDC to help facilitate cooperation among countries in Africa for the procurement and delivery of vaccines. Research published by the World Bank in April 2021 identified that multilateral action, including action by the African Union on vaccines that could supply enough vaccine to cover 60 percent of population by March 2022 (Agarwal and Reed 2021). It was not until June 2021, however, following months of discussion, that the World Bank announced that it was partnering to support the Africa Vaccine Acquisition Task Team of the Africa CDC with resources to help countries access vaccines, as a complement to COVAX. The formal announcement of the World Bank and Africa Vaccine Acquisition Task Team partnership was the culmination of months of work invested by World Bank management (World Bank 2021h). Earlier partnership on vaccines with the Africa CDC would have helped countries in Africa with limited capacities to act independently to aggregate demands and take earlier actions to pool efforts to procure and plan for vaccines across countries. The regional project analysis (appendix G) exposes the strong added value of support to regional organizations for engaging political leadership and helping countries plan and cooperate on procurement and technical actions for disease response, which could have been extended to vaccines. The Africa CDC partnership was key to help countries such as Mozambique, which has fully vaccinated 42 percent of its population (World Bank 2022a; box 4.6).1

World Bank teams supported planning for vaccines in countries in collaboration with partners. Work through the Access to COVID-19 Tools Accelerator at the country level provides an example of global partners aligning to coordinate planning, tracking, and diagnostics of vaccines at the country level. In countries, World Bank teams engaged in committees to plan and track vaccine support. In Djibouti, Honduras, the Philippines, and Tajikistan, the World Bank coordinated with partners to prepare for early vaccine deployment. In Mozambique, the Philippines, and Tajikistan, the World Bank supported digital tracking systems for vaccine rollout and communication about vaccines. More early emphasis could have been put on communication plans and activities to strengthen vaccine delivery in countries.

The World Bank’s MPA financing for vaccines provided prompt country-level support that coincided with the approval of the first vaccines, but it took time to develop country vaccine projects. In October 2020, the World Bank approved additional financing for vaccines of $12 billion for the MPA, which later grew to $20 billion. The financing was in anticipation of the upcoming emergency approval of the first COVID-19 vaccines by WHO (including $6 billion for IDA, of which $294.97 million was grant and $6 billion for International Bank for Reconstruction and Development countries). The first vaccines to receive emergency approval by WHO were Pfizer-BioNTech on December 31, 2020; Johnson & Johnson on March 12, 2021; and Moderna on April 30, 2021. The financing aimed to support the full vaccination of 1 billion people, or about 20 percent of country populations. The 20 percent coverage corresponded to the WHO Allocation Framework target for priority immunization that focused first on frontline health workers and caregivers, then the elderly and younger people with underlying conditions, which place them at higher risk for COVID-19 (WHO 2020a). The MPA emphasized community engagement and risk communication, which were limited in the early COVID-19 response. However, the approval of MPA vaccine projects in countries took time because a second step of the World Bank Board decision was required for the first five projects—after the restructuring of a project in Lebanon to purchase vaccines (January 20, 2021), the first five MPA vaccine projects were approved by the World Bank Board in Cabo Verde, Mongolia, and Tajikistan (February 1, 2021) and in Afghanistan and Nepal (March 18, 2021). Projects of $100 million and more also required board approval; Bangladesh, Ethiopia, the Philippines, and Tunisia were approved in March 2021. After these projects, approvals in other countries increased quickly in subsequent months, with vaccine projects approved in 19 countries by April 30, 2021.

Vaccine supply constraints on global markets and safeguards were challenging for the first MPA vaccine projects. Without earlier advance market commitments at the global level, countries with limited resources were often unable to access global supplies of vaccines, despite the well-designed and prompt country-level financing of the MPA. Of the countries with less than 20 percent COVID-19 vaccine coverage, 89 percent are IDA recipients, and 75 percent are in Africa. Vaccine supply remained constrained well into 2021, in part as a result of manufacturing issues and the emergence of the Delta variant that led to export restrictions on India’s vaccine supply. In 2021, the MPA financing for vaccines helped countries access vaccines from manufacturers and through donations, COVAX, and the Africa CDC, but supplies were often limited, and timing of receiving vaccines from different sources was difficult to manage and link to campaigns. Early use of MPA financing to procure vaccines in the first months of 2021 was also slowed by added safeguard policies. The procurement of vaccines needed to follow additional World Bank guidance and safeguards beyond those of WHO. The operational process review and case studies (appendix C) acknowledged the rationale of caution in financing new, unproven vaccines. However, this guidance meant that in early 2021, World Bank vaccine projects could not process government requests to pool vaccine purchases with partners or often support the distribution of donated vaccine resources. After several months, the World Bank adjusted its actions (on April 16, 2021), recognizing the challenge, and aligned with WHO guidelines and Gavi, the Vaccine Alliance. A waiver was also provided for the first vaccine project to encounter this challenge in Lebanon.