This evaluation assesses the quality of the World Bank’s early response to the COVID-19 crisis and the initial steps toward recovery, focusing on the health and social response. It concentrates on the relief stage and support to restructure systems in the first 15 months of the pandemic (February 1, 2020, to April 30, 2021) in 106 countries. A parallel Independent Evaluation Group evaluation looks at the World Bank Group support to address the economic implications of the pandemic. To assess the quality of the response, the evaluation is guided by a theory of action that synthesizes evidence in three dimensions: relevance of support to the needs of countries; implementation, learning, and adjustment; and operational policy and partnerships to support smooth responses in countries. As the response is ongoing, the evaluation does not assess effectiveness but considers early results and pathways that are expected to lead to outcomes.

Main Findings

In a context of high uncertainty, the World Bank delivered a response of unprecedented scale and speed. The immediate support was particularly swift in the most vulnerable countries. In the first 15 months, the World Bank provided financing of an estimated $30 billion for the health and social response in 106 countries with high or medium vulnerability to human capital and development losses. Support to small states, less-prepared countries, and fragile and conflict-affected situations was emphasized. About 20 percent of financing was disbursed in the first months of 2020, and 40 percent was disbursed by April 2021. Staff and clients worked long hours to deliver new and repurposed operations, all while learning to use remote connectivity tools and adapting to home-based work and personal stresses.

Relevance of Support to Country Needs

The evaluation looks at how well the World Bank responded to the immediate health threat of COVID-19, how well it focused on protecting vulnerable groups against human capital losses, and how well it integrated institutional strengthening in the relief stage to help sustain preparedness and resilience postcrisis.

The World Bank support was relevant to the needs of countries and well aligned with most emergency areas in their COVID-19 responses. Emergency support expanded critical health services to prevent and control the spread of disease, including infection prevention and control, case management, surveillance, and provision of laboratories. The support prioritized social protection for poor and vulnerable people. For example, Djibouti, Honduras, India, Senegal, and Tajikistan expanded emergency health and social protection actions through World Bank operations. World Bank support in countries aligned well with national COVID-19 plans of governments, which coordinated emergency support of development partners to the response.

World Bank support addressed country needs most comprehensively where earlier work on human capital had built preparedness and where cross-sectoral coordination among Global Practices (GPs) and sectors in countries was stronger. Knowledge and relationships developed before COVID-19 helped reorient country portfolios in human development and other sectors to accommodate newly emerged needs. For example, in Uganda, the response built on existing relationships in health, education, water, agriculture, and nutrition. In the Philippines, new relationships needed to be developed in health, initially slowing the early response, while work before COVID-19 in social protection and community development enabled the rapid expansion of cash transfer programs and support in communities. Coordination across sectors was weak in most countries. However, where coordination was stronger (for example, in India and Senegal), it helped quickly mobilize a range of GPs and sectors in the country to address needs related to testing, surveillance, laboratories, social protection, child learning, and nutrition, and involved women’s groups and the informal sector.

The early months of the World Bank response had a strong emergency focus, followed in about half of countries with efforts to protect human capital. The World Bank’s knowledge work on gender, epidemic preparedness, supply chains, social protection, and behavior change communication helped prioritize actions in some countries (Djibouti, Honduras, India, and Uganda). Strong government leadership helped some countries rapidly adapt World Bank support to both emergency and human capital needs. In the remaining half of countries, less attention was given in the first 15 months to continuing maternal and child health and education services, protecting women and girls from the shock of COVID-19, and engaging communities. The challenge of responding to urgent needs while protecting human capital was especially acute in countries with weaker systems for rapid health response and extensive human capital vulnerabilities (such as Chad and Niger).

Integrating institutional strengthening in the early COVID-19 response helped focus on sustaining public health preparedness and building resilience in health, education, and social protection systems. In more than 90 percent of countries, institutional strengthening was part of World Bank support. For example, in countries such as India, the Philippines, and Tajikistan, the World Bank helped strengthen and rapidly expand social protection systems, often to a national scale. In Djibouti and Uganda, extensive support in education helped develop and strengthen remote learning networks. Honduras emphasized early health support to strengthen laboratories. However, most countries still need strategies to sustain preparedness and ensure systems resilience after the crisis. Regional disease-focused projects, such as in Senegal, Zambia, and the countries of the Organisation of East Caribbean States, often helped countries to put better strategies in place for sustaining public health preparedness and to strengthen capacities in areas such as laboratories, testing, and case management.

Early Successes, Challenges, Learning, and Adjustment

The evaluation examines how well the World Bank supported implementation and adjustment to ensure a strong response. It looks at how well the World Bank supported countries to achieve early results, built on past lessons and evidence, and introduced innovation. It also examines how the World Bank used dialogue and coordination, knowledge sharing through regional projects, and data to inform decisions and adjust the response.

Although too early to observe outcomes, case studies provide promising evidence of early outputs that are key to satisfactory implementation and a good indicator that positive outcomes can be expected. Examples include the rapid expansion of critical health services, such as COVID-19 testing, social protection benefits, and remote learning for children. These interventions likely helped reduce the health threat of COVID-19 and protect human capital. About 40 percent of countries had projects that included monitoring data, and a mix of interventions that provide critical health services for disease prevention and control and limited coverage of interventions to protect human capital and engage communities.

Broadly, the World Bank used its experience from past crises and existing knowledge about effective interventions. Most health projects built on past lessons and incorporated effective disease prevention and control interventions. For example, countries received widespread support for laboratories and infection prevention and control for COVID-19. At the same time, support to local government and service providers, community-based interventions, and support to address gender equality (such as psychosocial care and sexual and reproductive health interventions) were limited, despite consistent evidence of effectiveness and lessons from past crises and risk communication. The focus on broad national response was strong, and attention to local-level implementation challenges in reaching vulnerable groups was less prominent. The burden of the pandemic on frontline health workers was heavy, yet innovations in service delivery during the crisis were rare. In education, while there was often a focus on local learning, case studies suggest that countries faced challenges in supporting teachers and vulnerable children to continue learning during the crisis (India, Mozambique, and Uganda).

In its effort to respond quickly and effectively, the World Bank innovated—its response included some form of innovation in more than 80 percent of countries. This evaluation found more than 200 examples of innovations supported by the World Bank in its COVID-19 response, such as for health communication and vaccine monitoring (Tajikistan), surveillance (Colombia), expanding cash transfers (the Democratic Republic of Congo), remote coaching of teachers (Lebanon), and multisector coordination (Haiti). In Senegal, community-based disease surveillance and multistakeholder engagement supported community health workers and volunteers to detect COVID-19 and report cases to health facilities and local government. In Mali, a new national call center provided advice for implementing COVID-19 protocols. Global partnerships and knowledge sharing by regional projects were useful to successfully promote innovation. For example, the World Bank’s Education Technology team helped countries to expand remote learning; regional disease-focused projects helped expand country innovations in infection prevention and control, point of entry control, testing, and surveillance.

The World Bank engaged in frequent dialogue with governments and partners to coordinate and adjust implementation. Supporting government coordination to implement responses at the national and subnational levels worked best where there were country-led structures that predated COVID-19. Coordination structures facilitated dialogue on emerging needs, strengthened responses, and involved frontline services and communities for oversight, learning, communication, and problem-solving. One Health structures, which coordinate multisectoral disease response actions, in Senegal and Zambia helped coordinate actions in health with other sectors. Subnational nutrition structures were key for COVID-19 messaging and for disease surveillance in Honduras, Senegal, and Uganda. Parent-teacher networks were important for supporting learning. New structures for coordination took time to set up during COVID-19, for example, in Haiti and the Philippines.

Regional projects facilitated knowledge sharing and were particularly helpful for countries with limited capacity to respond independently to COVID-19. Regional projects supported technical cooperation (such as for planning and reporting on the response) between ministries and public health institutes, encouraged leadership, developed human capacity, and coordinated technical sharing and financing for COVID-19 responses in countries. Longer-running regional projects had more established networks, which had successfully built some preparedness before the pandemic to support COVID-19 responses, although even newer regional projects added value, mainly through convening and technical and learning support. The Economic Community of West African States was wellprepared to support countries during COVID-19, largely thanks to earlier support under the Regional Disease Surveillance Systems Enhancement Project. Despite being a newer organization, the Africa Centres for Disease Control and Prevention, supported by a World Bank regional project, quickly developed convening structures in Africa, such as for collaboration for disease surveillance, testing, and vaccines. The Organisation of East Caribbean States Regional Health Project also quickly coordinated support for testing and case management.

Few countries possessed real-time data systems and adequate data to inform decisions and adapt the response. Where they existed, diagnostics (Djibouti), geo-enabled monitoring (Tajikistan), iterative beneficiary monitoring, short messaging systems (Lesotho), online surveys (Tunisia), and dashboards (Colombia) supplied timely data to inform decisions, monitor behavioral change, and adjust actions. Where available, frequent data on the quality of health and education services in communities were critical for course corrections. Honduras and Uganda used remote supervision systems to monitor and improve local nutrition services during COVID-19. Tajikistan and Zambia used short messaging systems to track vaccine services and communicate with teachers. Djibouti, India, the Philippines, Senegal, and Tajikistan used real-time survey data to adjust social protection responses.

Operational Policies and Partnerships

The assessment of the operational policies and partnership looks at how well the World Bank’s internal coordination, instruments for financing the COVID-19 response, and internal systems for reporting and monitoring supported the response. It looks also at the World Bank’s financing and technical partnership, including support of the Pandemic Emergency Financing Facility and support to vaccine financing.

At the onset of the pandemic, Bank Group senior management demonstrated strategic and agile decision-making. Bank Group senior management articulated its approach early in March 2020 and delivered an Approach Paper to its Executive Directors in June 2020. This included front-loading International Development Association spending allocations and seeking an unprecedented International Development Association replenishment a year ahead of schedule, activating the International Bank for Reconstruction and Development’s crisis buffer to release additional financing, and aligning with the World Health Organization technical guidance on health issues. Within the World Bank, the Emergency Operations Center facilitated good internal coordination across GPs and operational support units, which was critical for action alignment and technical problem-solving. Policy guidance and knowledge sharing in GPs helped guide World Bank teams’ design projects in the early months of the response. Country portfolio reviews led by World Bank country management facilitated coordination of support across GPs and project teams in countries. To quickly process projects, managers in the health sector mobilized surge capacity involving retirees, exchanging staff, and increasing the responsibilities of country office staff. Wider engagement of GPs outside Human Development could have drawn on more staff resources and financing to help countries and coordinate efforts to process project support in the early months of the response.

At the country level, having a pre–COVID-19 World Bank program with a good mix of instruments, including crisis instruments that could support timely financing in the first weeks of the crisis, facilitated a swift response. Crisis instruments, repurposed projects, regional projects, trust funds, and grants, where available in country program portfolios, helped rapid financing and just-in-time assistance in the early weeks and months of the crisis. Other instruments built on this support but often took longer to process: the Multiphase Programmatic Approach (MPA) was key to expanding new lending in more vulnerable countries for the health emergency response; development policy financing provided important funding for early systems strengthening in areas where it could achieve quick wins by building on previous policy dialogue on human capital; and Pandemic Emergency Financing Facility grants supported COVID-19 plans and coordination with United Nations partners, although the small amounts of funding took time to process. The early responses in Senegal and Uganda relied on crisis instruments, repurposed projects, and trust funds, which were complemented by development policy financing, Pandemic Emergency Financing Facility, and MPA support once available. Tajikistan used repurposed projects for its early COVID-19 response and then used the MPA financing in health when it became available.

The World Bank introduced operational flexibility, which facilitated rapid processing of new financing for the MPA. This included shortened clearance times and delegation of approvals. The first MPA projects disbursed in about two months compared with about five months in previous crises. This quick timing was important because there was less reliance on additional financing compared with previous crises. Other new investment project financing projects took about five months to disburse, but in some countries, projects disbursed in less than one month. The procurement of medical goods early in the response also happened rapidly; from the first month of the response, personal protective equipment, test kits, and medical equipment were procured for emergency use in countries by using World Bank–facilitated processes and hands-on assistance or enabling governments to use emergency procedures in projects. Despite the extensive support of safeguard teams, the new Environmental and Social Framework was challenging for new projects in the first months of the crisis, given that ministries were overwhelmed, and it required new learning. Requirements of citizen engagement and gender could have benefited from more hands-on assistance to help teams.

It was challenging to collect timely data to report on the progress of support and track and coordinate procurement. Integrated reporting of data on various parts of the World Bank’s country-level COVID-19 response was important for discussions with governments, World Bank teams, partners, and headquarters. In India and other countries, the World Bank country office often lacked timely data to track implementation of projects to inform coordination. A key challenge was the difficulty in coordinating government procurement requests with other development partners in countries so the same items were not purchased. The tracking of goods—from ordering to shipping to arrival in health facilities—was also crucial though rare. Tracking was challenging, given the limited emergency preparedness of procurement systems in countries and lack of remote monitoring mechanisms.

Having well-established partnerships with development organizations in place before the COVID-19 pandemic facilitated rapid action. For example, the Global Partnership for Education, where available, helped quickly expand education support for children (such as in Uganda). In Mozambique, Senegal, and Uganda, partnership with the Global Financing Facility helped expand maternal and child health services and risk communication, though actions could have been quicker. In Tajikistan, the Global Partnership for Social Accountability supported efforts to involve civil society to monitor the COVID-19 response, and partnership with Gavi, the Vaccine Alliance supported early planning to access vaccines. Existing country-level development partnerships enabled coordinated financing and actions for the response (for example, in India and the Philippines). Collaboration with nongovernmental organizations and the private sector in World Bank projects (such as in Belize, India, Peru, and Togo) helped expand community-based implementation, innovation, and use of technology and digital payments in social protection.

In the uncertain early months, the World Bank made good efforts in engaging with partners to help prepare countries to deliver vaccines and expedite access, but the World Bank lacked an instrument to rapidly facilitate advance market commitments. In the first months of the pandemic, the Health, Nutrition, and Population GP convened global partners to explore ways to help low-income countries access vaccines. This was followed by intense internal dialogue about how the World Bank could best support vaccine readiness and access, focusing on supporting country-level efforts for vaccines, given the lack of a global instrument to help finance advance market commitments. Partnerships worked well at the country level, and the MPA financing for vaccines was timely. But implementation of vaccine support was initially slow because countries had limited health systems capacities to support delivery, and they often could not access vaccines early in the response. The key was having access to financing for advance resource commitments, pooling resources with other partners in countries to support procurement, and aligning efforts in countries for vaccine safety and delivery. Earlier engagement with partners—namely, the COVID-19 Vaccines Global Access initiative and the African Vaccine Acquisition Trust—could have helped ensure advanced vaccine supplies for countries but also facilitated earlier preparedness and communication about vaccines.

Overall, the quality of the early World Bank response was good. Looking ahead, a number of areas need attention by the World Bank and its clients: better preparedness of countries to deliver emergency services that reach local levels; more resilient systems in countries to protect health, education, and gender equality; improved support for cross-sectoral coordination; data for managing quality implementation; regional learning and cooperation; and stronger internal preparedness to respond quickly in a crisis, including coordination with partners.

Recommendations

The findings from the evaluation inform four recommendations for ensuring stronger future preparedness.

Recommendation 1. Use the World Bank’s crisis recovery efforts to strengthen the resilience of essential health and education services to ensure that human capital is protected in a crisis.

Proposed Actions

• In health, build on innovations developed during COVID-19 to help countries strengthen telehealth and other platforms for continuing essential health services in an emergency. Help countries improve the quality of frontline services, including the availability of data to inform decisions for quality improvements. Services could be improved to better manage supplies, deliver vaccines, support health workers to deliver effective care, and ensure infection prevention and control measures. The availability and use of feedback from beneficiaries and coverage of vulnerable groups are also important. The World Bank could also help develop new capacities to deliver services, such as in psychosocial care.
• In education, draw on evidence and innovations of the COVID-19 response to strengthen platforms for continuous learning in a crisis. Strengthen community networks that have been established to support learning. To avoid learning losses, facilitate knowledge building to uptake effective approaches to help children in and out of school catch up. Help countries increase the reach to vulnerable groups that may have been missed by remote learning. Strengthen monitoring of beneficiary feedback on the quality of learning.

Recommendation 2. Apply a gender equality lens to health and social crisis response actions across sectors.

Proposed Actions

• Develop actions across sectors (in health, education, urban, and social protection) for protecting women and girls from shocks, which can be drawn on in a crisis response. This is especially important in countries with high needs for addressing gender equality. Examples of areas to support gender equality include psychosocial support, sexual and reproductive health, income and asset accumulation, and community engagement.

Recommendation 3. Help countries strengthen regional cooperation and crisis response capacities for public health preparedness.

Proposed Actions

• Support regional organizations to facilitate cooperation, political leadership, and technical learning, especially in Africa. Such support could help strengthen preparedness in countries and regional mechanisms for crisis response, facilitate financing and technical partnerships, encourage innovation, and expand evidence to scale up effective approaches. Regional support could also facilitate evidence-based and data tools to help countries monitor crisis response actions.
• Help countries strengthen national and subnational platforms to coordinate and deliver crisis interventions, such as One Health platforms, with greater emphasis on critical health services and demand-side activities, such as citizen engagement. At the national level, invest in platforms that coordinate action and prepare various sectors to take on specific roles in crisis. At the subnational level, invest in platforms that can reach local government and communities for disease surveillance, risk communication, delivery of health and social services, and monitoring support.

Recommendation 4. Build on the COVID-19 experience to strengthen the World Bank’s internal crisis preparedness so that it has the tools and procedures ready to respond in future emergencies.

Proposed Actions

• Review and expand operational flexibilities for processing new projects in crises and develop guidance on the effective use of instruments at different stages of crisis response. The World Bank could also explore innovative ways to strengthen the use of crisis instruments in countries, such as through support to communities, and expand guidance on hands-on assistance for citizen engagement and gender, learning from the provision of such support in procurement.
• Expand and strengthen the World Bank’s partnerships and instruments to enable coordinated financing, advance market commitments, and technical support that will help countries strengthen crisis preparedness. The partnerships could be at the global, regional, and country levels. They could include technical partnerships to expand knowledge for quality implementation of preparedness activities, partnerships with nongovernment and the private sector to support community-based implementation, feedback on services and use of technology, and global partnership for aligning financing, plans, and guidance to support countries.
• Strengthen tools to allow for the integrated management and frequent reporting of monitoring data on projects in World Bank portfolios.