The evaluation answers the following question: What has been the quality of the World Bank’s early COVID-19 response in countries in terms of saving lives and protecting poor and vulnerable people? The focus is on the first 15 months of the pandemic and the large-scale and rapid actions that took place during these months in a very uncertain global context.

Outcomes are not assessed; rather, the focus is on how the quality of the early response, design, processes, and outputs supported pathways to outcomes.

The evaluation’s purpose is also to draw lessons from the World Bank’s support in the early COVID-19 response to inform recovery efforts and future support for crisis preparedness.

Response

The World Bank Group quickly launched a large-scale response to help countries address both the health threat and the social and economic impact of the COVID-19 crisis. The support comprised three stages (relief, restructuring, and recovery) and four pillars (saving lives, protecting poor and vulnerable populations, ensuring sustainable business growth and jobs, and strengthening institutions for recovery). Country support was to be tailored to address needs and priorities across these areas. Institutional strengthening was to be undertaken from early in the relief stage to ensure sustained support to countries and maintain a clear route toward their longer-term development priorities. Attention to gender equality, digitalization, monitoring, evaluation, learning, and encouraging innovations was intended to cut across all the pillars (World Bank Group 2020b). The Bank Group support to a country was often part of a national COVID-19 response plan, developed in collaboration with the government, partners, and other country stakeholders.

In a highly uncertain context, the response was notably swift and unprecedented, with support provided to more than 100 countries: in February 2020, the Bank Group committed $160 billion in financing for the COVID-19 response in fiscal years 2020 and 2021. The response was aligned with global actions of the World Health Organization (WHO) and other partners (figure 1.1). The total financing included about $76 billion of World Bank commitments, of which about half supported the health and social response—the focus of this evaluation—in three of the four pillars (saving lives, protecting poor and vulnerable populations, and strengthening institutions for recovery). Other financing was for the economic response in sectors such as agriculture and governance and for the International Finance Corporation response, which are outside the scope of this evaluation. This all took place in a context with much uncertainty about the virus, vaccines, and how to best respond quickly and at scale, with information on the situation evolving daily. Key elements of the response included support through a Multiphase Programmatic Approach (MPA), new projects prepared through a fast-track facility, repurposing of existing projects, grant support from the Pandemic Emergency Financing Facility (PEF), and activation of existing crisis instruments embedded in the portfolio. Although the pandemic continues to destroy lives with waves of infection occurring at different times and intensities across countries, even with deployment of effective vaccines, attention is moving toward coping with COVID-19 endemism, protecting vulnerable populations, and restructuring systems for recovery. This will shift the priorities in social and health sectors toward a return to stability and building back better to ensure future preparedness and protection of human capital.

Evaluation Purpose and Scope

Purpose

The evaluation provides an early assessment of the quality of the World Bank’s COVID-19 health and social response to save lives and protect people living in poverty. Its purpose is to draw lessons to inform ongoing and future support for crisis preparedness and response. As the response is ongoing, the evaluation pays attention to processes, outputs, and pathways to outcomes, focusing on the relief stage and early support to restructure systems that can inform learning for recovery and future pandemic and crisis preparedness.

Figure 1.1. Global Milestones and Timeline of World Bank Group Response

Image

Share Tweet Facebook WhatsApp Linkedin Email

Figure 1.1. Global Milestones and Timeline of World Bank Group Response

Sources: Barış et al. 2021; Dixon et al. 2021; Independent Panel for Pandemic Preparedness and Response 2021; KNOMAD 2021; Ravelo and Jerving 2022; Saavedra et al. 2020; WHO 2020a, 2020b, 2021b, 2022; World Bank 2020a, 2020b, 2020c, 2020d, 2020e, 2020f; World Bank Group 2021a, 2021b.

Note: ACT = Access to COVID-19 Tools; AMC = advance market commitment; AVAT = African Vaccine Acquisition Trust; COVAX = Country Access to COVID-19 Vaccines; G7 = Group of Seven; GRID = Green, Resilient, and Inclusive Development; HNP = Health, Nutrition, and Population; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; IFC = International Finance Corporation; IMF = International Monetary Fund; MPA = Multiphase Programmatic Approach; PEF = Pandemic Emergency Financing Facility; SPJ = Social Protection and Jobs; UNGA = United Nations General Assembly; UNGASS = United Nations General Assembly Special Session; UNICEF = United Nations Children’s Fund; WHO = World Health Organization.

Scope

The evaluation answers the following question: What has been the quality of the World Bank’s early COVID-19 response in countries in terms of saving lives and protecting poor and vulnerable people? The evaluation uses three lines of inquiry to answer this question (box 1.1).

The evaluation focus is the health and social response during the first 15 months of the COVID-19 pandemic, from February 1, 2020, to April 30, 2021, and a portfolio of 106 countries. The evaluation looks to learn from the World Bank’s health and social support to countries most vulnerable to reversal of development and human capital gains because of COVID-19 (see appendix B for the portfolio identification). Five World Bank Global Practices (GPs) led the early support to countries: Health, Nutrition, and Population; Social Protection and Jobs; Education; Urban, Disaster Risk Management, Resilience, and Land; and Macroeconomics, Trade, and Investment. A concurrent Independent Evaluation Group (IEG) evaluation assesses the economic response to COVID-19.

The health and social support evaluation portfolio consists of an estimated $30 billion of commitments to save lives and protect poor and vulnerable people in vulnerable countries, of which about $11 billion (one-third) was committed by May 2020 (appendix B).1 This includes $27 billion in operational financing ($15 billion International Development Association [IDA] and $14 billion International Bank for Reconstruction and Development), $1.5 billion in trust funds, and $60 million in advisory services and analytics (ASA) commitments. Forty percent of this was disbursed in the first 15 months of the response, and about 20 percent was disbursed in the first three months, by May 2020. It is also estimated that there was $1.54 billion in Contingency Emergency Response Component (CERC) commitments from other GPs allocated to the health and social response not covered by the portfolio. The Human Development GP led these commitments. During the early response, Health, Nutrition, and Population had five times more projects approved and managed about 600 percent more in annual allocations, spread across small projects (about 43 percent smaller on average than other health projects) with short durations of two to three years. Social Protection and Jobs almost doubled the number of projects approved, whereas Education had about 56 percent more projects. For other GPs, lending activity increases were more limited during the early response to COVID-19, whereas lending in areas such as Agriculture and Food increased in the second year of the response.

The evaluation portfolio focuses on countries with moderate to high vulnerability to development losses because of the impact of COVID-19.2 It emphasizes support to less-prepared countries, small states, countries with fragile and conflict-affected situations (FCS), and countries at risk of human capital losses (appendix B). Most financing commitments were allocated to less-prepared countries with pressing needs. Commitments in these countries included support of new projects and repurposing of existing projects and ASA to support the crisis response. Small states received about $26 million per million population in the COVID-19 response compared with about $6 million for other countries. FCS countries received $9 million per million population. New project support focused on countries with lower levels of human capital in Africa and South Asia that included countries with high vulnerability to development losses as a result of COVID-19.

Evaluation Design

Conceptual Framework

The conceptual framework defines the thematic areas and stages of support within the scope of the early COVID-19 response assessed by the evaluation (figure 1.2 and box 1.2). The framework is based on the Bank Group’s COVID-19 response framework (World Bank Group 2020b). The conceptual framework supports a theory-based approach that models the interlinked elements of the health and social response, with the current evaluation focusing on the early World Bank response in the relief and initial restructuring stages.

Figure 1.2. Conceptual Framework for COVID-19 Health and Social Response

Image

Share Tweet Facebook WhatsApp Linkedin Email

Figure 1.2. Conceptual Framework for COVID-19 Health and Social Response

Source: Independent Evaluation Group. Adapted from World Bank Group 2020b to focus on the health and social aspects of the COVID-19 response.

Note: IPC = infection prevention and control.

Methodology

The evaluation adopts a multilevel analysis and a mixed methods approach that combines quantitative and qualitative evidence. It follows a consultative approach to inform the analyses and a modular approach to share preliminary findings. Throughout the evaluation, there was engagement with World Bank GP management and project teams, operational support units, country management, and technical experts to discuss analyses, share preliminary findings, and receive feedback. These consultations are important because the evaluation focuses on an active and evolving portfolio.

Theory of Action

The evaluation’s theory of action outlines the dimensions for the assessment of the quality of the World Bank’s early COVID-19 response (figure 1.3). The theory of action aligns with the evaluation questions and complements the conceptual framework. It posits three interrelated areas that define the quality of the response, each with specific dimensions for which the evaluation gathered and triangulated evidence:

• Support to needs looks at the relevance of the World Bank’s large-scale early response to help address the immediate health threat of COVID-19 and to protect vulnerable groups against human capital losses.
  • Dimensions: Responsiveness of World Bank support to health and social needs of countries, addressing gender, building on existing human capital capacities, alignment with COVID-19 plans in countries, reorientation of projects in the portfolio, prioritization of support in the portfolio to focus on key areas within needs and vulnerable groups, use of knowledge work to inform needs, and integrating institutional strengthening and support to build resilient systems for recovery
• Implementation and learning looks at factors that facilitated and hindered the World Bank early response in countries, which can provide proxy evidence of early results, and how there has been learning and adjustment to strengthen the response.
  • Dimensions: Implementation status of the response, factors facilitating implementation, early results, anchoring of support in lessons and evidence from past crises, and emphasis on innovation, learning, dialogue and coordination with the government and other stakeholders in countries, use of data and other inputs to make course corrections to ensure a strong mix of support to countries, and regional knowledge sharing
• Operational policies and partnerships looks at how well World Bank internal process, instruments, and partnerships supported a smooth and speedy early response.
  • Dimensions: World Bank internal coordination, the mix of instruments supporting the response, streamlined operational processes, internal reporting and monitoring, and development partnerships including PEF and vaccines support.

Figure 1.3. Theory of Action to Assess the Quality of the Early COVID-19 Response

Image

Share Tweet Facebook WhatsApp Linkedin Email

Figure 1.3. Theory of Action to Assess the Quality of the Early COVID-19 Response

Source: Independent Evaluation Group.

Levels of Analysis

The assessment of quality and lessons arise from triangulating evidence gathered by different evaluation components at the country, portfolio, and corporate levels. The evaluation uses a mix of methodological applications to ensure construct, internal, and external validity and reliability of findings through a transparent methodological design, with clear justification of choices made (see appendix A for the evaluation methodology). Box 1.3 describes the evaluation components.

Limitations

An important limitation of the evaluation is the dynamic nature of the COVID-19 situation and World Bank response, and the overlap of the evaluation with the ongoing response implementation. The World Bank was adapting the response during implementation to improve its actions, given the evolving and uncertain global and country-level contexts of COVID-19. Moreover, interviews were challenging because of the high number of COVID-19 cases in some countries, the burden of the pandemic on health sector personnel, and illness and personal losses of interviewees as a result of COVID-19. Although the evaluation analyses estimate early support and financing in the countries covered by the portfolio, a rapid update of the portfolio was done at the end of the evaluation to gauge shifts in support as the portfolio continued to evolve throughout the evaluation timeline. A future later-stage evaluation is proposed to look at the effectiveness of the response. Appendix A also outlines limitations of specific methods of the evaluation.

Structure

The report structure is based on the theory of action of the evaluation. Each chapter highlights evidence on early support to countries to effectively respond to the health and social shocks of COVID-19 and to start a process to strengthen systems and policy for better crisis preparedness and protection of human capital. Chapter 2 looks at the extensive scale of the World Bank support and its relevance to needs of countries. Chapter 3 covers the implementation successes and challenges of the World Bank’s early support to countries, which can point to early results, and how the World Bank has adapted and learned during implementation. Chapter 4 covers how well operational policies and partnerships facilitated a smooth and speedy response. Chapter 5 synthesizes key areas of learning from the evaluation, which can inform future preparedness, and presents recommendations for the way forward.