Back to cover

The World Bank’s Early Support to Addressing COVID-19

Chapter 1 | Introduction

The evaluation answers the following question: What has been the quality of the World Bank’s early COVID-19 response in countries in terms of saving lives and protecting poor and vulnerable people? The focus is on the first 15 months of the pandemic and the large-scale and rapid actions that took place during these months in a very uncertain global context.

Outcomes are not assessed; rather, the focus is on how the quality of the early response, design, processes, and outputs supported pathways to outcomes.

The evaluation’s purpose is also to draw lessons from the World Bank’s support in the early COVID-19 response to inform recovery efforts and future support for crisis preparedness.

Response

The World Bank Group quickly launched a large-scale response to help countries address both the health threat and the social and economic impact of the COVID-19 crisis. The support comprised three stages (relief, restructuring, and recovery) and four pillars (saving lives, protecting poor and vulnerable populations, ensuring sustainable business growth and jobs, and strengthening institutions for recovery). Country support was to be tailored to address needs and priorities across these areas. Institutional strengthening was to be undertaken from early in the relief stage to ensure sustained support to countries and maintain a clear route toward their longer-term development priorities. Attention to gender equality, digitalization, monitoring, evaluation, learning, and encouraging innovations was intended to cut across all the pillars (World Bank Group 2020b). The Bank Group support to a country was often part of a national COVID-19 response plan, developed in collaboration with the government, partners, and other country stakeholders.

In a highly uncertain context, the response was notably swift and unprecedented, with support provided to more than 100 countries: in February 2020, the Bank Group committed $160 billion in financing for the COVID-19 response in fiscal years 2020 and 2021. The response was aligned with global actions of the World Health Organization (WHO) and other partners (figure 1.1). The total financing included about $76 billion of World Bank commitments, of which about half supported the health and social response—the focus of this evaluation—in three of the four pillars (saving lives, protecting poor and vulnerable populations, and strengthening institutions for recovery). Other financing was for the economic response in sectors such as agriculture and governance and for the International Finance Corporation response, which are outside the scope of this evaluation. This all took place in a context with much uncertainty about the virus, vaccines, and how to best respond quickly and at scale, with information on the situation evolving daily. Key elements of the response included support through a Multiphase Programmatic Approach (MPA), new projects prepared through a fast-track facility, repurposing of existing projects, grant support from the Pandemic Emergency Financing Facility (PEF), and activation of existing crisis instruments embedded in the portfolio. Although the pandemic continues to destroy lives with waves of infection occurring at different times and intensities across countries, even with deployment of effective vaccines, attention is moving toward coping with COVID-19 endemism, protecting vulnerable populations, and restructuring systems for recovery. This will shift the priorities in social and health sectors toward a return to stability and building back better to ensure future preparedness and protection of human capital.

Evaluation Purpose and Scope

Purpose

The evaluation provides an early assessment of the quality of the World Bank’s COVID-19 health and social response to save lives and protect people living in poverty. Its purpose is to draw lessons to inform ongoing and future support for crisis preparedness and response. As the response is ongoing, the evaluation pays attention to processes, outputs, and pathways to outcomes, focusing on the relief stage and early support to restructure systems that can inform learning for recovery and future pandemic and crisis preparedness.

Figure 1.1. Global Milestones and Timeline of World Bank Group Response

Image

Figure 1.1. Global Milestones and Timeline of World Bank Group Response

 

Sources: Barış et al. 2021; Dixon et al. 2021; Independent Panel for Pandemic Preparedness and Response 2021; KNOMAD 2021; Ravelo and Jerving 2022; Saavedra et al. 2020; WHO 2020a, 2020b, 2021b, 2022; World Bank 2020a, 2020b, 2020c, 2020d, 2020e, 2020f; World Bank Group 2021a, 2021b.

Note: ACT = Access to COVID-19 Tools; AMC = advance market commitment; AVAT = African Vaccine Acquisition Trust; COVAX = Country Access to COVID-19 Vaccines; G7 = Group of Seven; GRID = Green, Resilient, and Inclusive Development; HNP = Health, Nutrition, and Population; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; IFC = International Finance Corporation; IMF = International Monetary Fund; MPA = Multiphase Programmatic Approach; PEF = Pandemic Emergency Financing Facility; SPJ = Social Protection and Jobs; UNGA = United Nations General Assembly; UNGASS = United Nations General Assembly Special Session; UNICEF = United Nations Children’s Fund; WHO = World Health Organization.

Scope

The evaluation answers the following question: What has been the quality of the World Bank’s early COVID-19 response in countries in terms of saving lives and protecting poor and vulnerable people? The evaluation uses three lines of inquiry to answer this question (box 1.1).

Box 1.1. Three Questions That Guide the Evaluation

1. What has been the relevance of the World Bank’s early COVID-19 response to addressing the needs of countries in saving lives and protecting poor and vulnerable people (that is, the diagnosis, design, and tailoring of interventions to country situations)?

2. What has facilitated or hindered implementation of the World Bank’s COVID-19 responses in countries, and how is the World Bank supporting learning and adjustments?

3. How well are operational processes, instruments, and partnerships supporting the World Bank’s COVID-19 responses in countries?

Source: Independent Evaluation Group.

The evaluation focus is the health and social response during the first 15 months of the COVID-19 pandemic, from February 1, 2020, to April 30, 2021, and a portfolio of 106 countries. The evaluation looks to learn from the World Bank’s health and social support to countries most vulnerable to reversal of development and human capital gains because of COVID-19 (see appendix B for the portfolio identification). Five World Bank Global Practices (GPs) led the early support to countries: Health, Nutrition, and Population; Social Protection and Jobs; Education; Urban, Disaster Risk Management, Resilience, and Land; and Macroeconomics, Trade, and Investment. A concurrent Independent Evaluation Group (IEG) evaluation assesses the economic response to COVID-19.

The health and social support evaluation portfolio consists of an estimated $30 billion of commitments to save lives and protect poor and vulnerable people in vulnerable countries, of which about $11 billion (one-third) was committed by May 2020 (appendix B).1 This includes $27 billion in operational financing ($15 billion International Development Association [IDA] and $14 billion International Bank for Reconstruction and Development), $1.5 billion in trust funds, and $60 million in advisory services and analytics (ASA) commitments. Forty percent of this was disbursed in the first 15 months of the response, and about 20 percent was disbursed in the first three months, by May 2020. It is also estimated that there was $1.54 billion in Contingency Emergency Response Component (CERC) commitments from other GPs allocated to the health and social response not covered by the portfolio. The Human Development GP led these commitments. During the early response, Health, Nutrition, and Population had five times more projects approved and managed about 600 percent more in annual allocations, spread across small projects (about 43 percent smaller on average than other health projects) with short durations of two to three years. Social Protection and Jobs almost doubled the number of projects approved, whereas Education had about 56 percent more projects. For other GPs, lending activity increases were more limited during the early response to COVID-19, whereas lending in areas such as Agriculture and Food increased in the second year of the response.

The evaluation portfolio focuses on countries with moderate to high vulnerability to development losses because of the impact of COVID-19.2 It emphasizes support to less-prepared countries, small states, countries with fragile and conflict-affected situations (FCS), and countries at risk of human capital losses (appendix B). Most financing commitments were allocated to less-prepared countries with pressing needs. Commitments in these countries included support of new projects and repurposing of existing projects and ASA to support the crisis response. Small states received about $26 million per million population in the COVID-19 response compared with about $6 million for other countries. FCS countries received $9 million per million population. New project support focused on countries with lower levels of human capital in Africa and South Asia that included countries with high vulnerability to development losses as a result of COVID-19.

Evaluation Design

Conceptual Framework

The conceptual framework defines the thematic areas and stages of support within the scope of the early COVID-19 response assessed by the evaluation (figure 1.2 and box 1.2). The framework is based on the Bank Group’s COVID-19 response framework (World Bank Group 2020b). The conceptual framework supports a theory-based approach that models the interlinked elements of the health and social response, with the current evaluation focusing on the early World Bank response in the relief and initial restructuring stages.

Figure 1.2. Conceptual Framework for COVID-19 Health and Social Response

Image

Figure 1.2. Conceptual Framework for COVID-19 Health and Social Response

Source: Independent Evaluation Group. Adapted from World Bank Group 2020b to focus on the health and social aspects of the COVID-19 response.

Note: IPC = infection prevention and control.

Box 1.2. The Logic of the Conceptual Framework

The conceptual framework outlines the elements of the World Bank’s health and social support for COVID-19, which are the focus of the evaluation. These elements are anchored in three pillars of the World Bank Group response (saving lives, protecting poor and vulnerable populations, and strengthening institutions for recovery). The framework expresses interlinked elements to respond to the health and social shocks of COVID-19 regarding:

  • The progression of the response through the three stages (relief, restructuring, and recovery)
  • The menu of areas that could be operationalized through World Bank support and tailored to needs in the country context
  • The types of results expected at each stage to respond to the health threat and protect human capital, from early outputs and processes to longer-term outcomes for recovery
  • The integration of institutional strengthening from the early response to improve capacities to manage crises and build more resilient systems to protect against future shocks, including supporting coordination and planning, expanding public health functions (such as disease surveillance), and strengthening local services
  • The cross-cutting areas important to effective implementation in a country (gender equality, improvements to digitalize systems, and monitoring, evaluation, and learning)

The evaluation assesses the World Bank’s health and social support to the early relief stage and initial restructuring stage of COVID-19. The relief stage concentrates on responding to the immediate health and social shocks of COVID-19. In health, relief stage is intended to ensure the supply of critical health services (infection prevention and control, case management, surveillance, and laboratories), early readiness for vaccines, and health risk communication. Continued access to essential health services for primary care of vulnerable groups was also important. The health response is intended to be coupled with tailored support to address social shocks as a result of COVID-19 and to protect accumulated human capital.

In the relief stage, the social emergency response aims to ensure social protection for poor and vulnerable groups, continued child welfare (access to learning and nutrition), psychosocial services for mental health, community and citizen engagement, and social cohesion support for the demand-side aspects of the response. Vulnerable groups include the elderly, people with underlying conditions or comorbidities, poor and marginalized populations, and women and children.

The restructuring stage can overlap with and follow the relief stage. Restructuring stage support seeks to strengthen systems and policies for public health preparedness and restoring human capital. The restructuring stage in the health response seeks to strengthen health systems, vaccine delivery, and essential health services. The social response seeks to improve social protection systems, education systems, and community resilience.

The recovery stage had yet to be reached in the early COVID-19 response, but relief efforts and some initial restructuring are intended to be put in place as building blocks to transition to this stage. The recovery stage is intended to apply the learning from the COVID-19 response to ensure pandemic-ready health systems; improve equity and inclusion through better access to health, education, and social services; and enhance policies that protect human capital.

Source: Independent Evaluation Group.

Methodology

The evaluation adopts a multilevel analysis and a mixed methods approach that combines quantitative and qualitative evidence. It follows a consultative approach to inform the analyses and a modular approach to share preliminary findings. Throughout the evaluation, there was engagement with World Bank GP management and project teams, operational support units, country management, and technical experts to discuss analyses, share preliminary findings, and receive feedback. These consultations are important because the evaluation focuses on an active and evolving portfolio.

Theory of Action

The evaluation’s theory of action outlines the dimensions for the assessment of the quality of the World Bank’s early COVID-19 response (figure 1.3). The theory of action aligns with the evaluation questions and complements the conceptual framework. It posits three interrelated areas that define the quality of the response, each with specific dimensions for which the evaluation gathered and triangulated evidence:

  • Support to needs looks at the relevance of the World Bank’s large-scale early response to help address the immediate health threat of COVID-19 and to protect vulnerable groups against human capital losses.
    • Dimensions: Responsiveness of World Bank support to health and social needs of countries, addressing gender, building on existing human capital capacities, alignment with COVID-19 plans in countries, reorientation of projects in the portfolio, prioritization of support in the portfolio to focus on key areas within needs and vulnerable groups, use of knowledge work to inform needs, and integrating institutional strengthening and support to build resilient systems for recovery
  • Implementation and learning looks at factors that facilitated and hindered the World Bank early response in countries, which can provide proxy evidence of early results, and how there has been learning and adjustment to strengthen the response.
    • Dimensions: Implementation status of the response, factors facilitating implementation, early results, anchoring of support in lessons and evidence from past crises, and emphasis on innovation, learning, dialogue and coordination with the government and other stakeholders in countries, use of data and other inputs to make course corrections to ensure a strong mix of support to countries, and regional knowledge sharing
  • Operational policies and partnerships looks at how well World Bank internal process, instruments, and partnerships supported a smooth and speedy early response.
    • Dimensions: World Bank internal coordination, the mix of instruments supporting the response, streamlined operational processes, internal reporting and monitoring, and development partnerships including PEF and vaccines support.

Figure 1.3. Theory of Action to Assess the Quality of the Early COVID-19 Response

Image
The theory of action diagram shows three interrelated areas that define the quality of the World Bank’s early response.

Figure 1.3. Theory of Action to Assess the Quality of the Early COVID-19 Response

Source: Independent Evaluation Group.

Levels of Analysis

The assessment of quality and lessons arise from triangulating evidence gathered by different evaluation components at the country, portfolio, and corporate levels. The evaluation uses a mix of methodological applications to ensure construct, internal, and external validity and reliability of findings through a transparent methodological design, with clear justification of choices made (see appendix A for the evaluation methodology). Box 1.3 describes the evaluation components.

Box 1.3. Evaluation Components at Each Level

At the country level:

  • Eight country case studies provided in-depth evidence on the COVID-19 response in specific contexts (Djibouti, Honduras, India, Mozambique, the Philippines, Senegal, Tajikistan, and Uganda). The team reviewed World Bank projects and analytic work, and interviewed staff, government officials, and representatives of civil society (appendix C).
  • A review of country situations was conducted to develop a heat map to understand the needs of countries early in the COVID-19 pandemic, analyze the extent of addressing these needs in countries in the portfolio, and identify factors that facilitated satisfactory implementation of World Bank support. The analysis used portfolio data on World Bank projects and advisory services and analytics supporting COVID-19 and publicly available health and social data on country indicators relevant to areas of the conceptual framework of the evaluation (appendix D).
  • A rapid review of the literature synthesized evidence on effective crisis interventions from systematic reviews and country studies of past epidemic and crises to benchmark World Bank support. The review synthesized existing evidence from the literature on 50 crisis interventions. The findings were used to understand the alignment of interventions supported by the World Bank’s early COVID-19 response with the available evidence base (appendix E).
  • A review of past crisis response projects benchmarked successes and challenges from these projects against the early COVID-19 response. The evaluation synthesizes lessons from 170 closed projects where the World Bank responded to crises in the past 20 years (appendix F).
  • A review of regional projects assessed the early results of disease-focused projects for COVID-19. Interviews and document review were used to understand the value added of four disease-focused regional projects to help advance early results of COVID-19 responses in countries (appendix G).
  • A stocktaking analysis identified innovations to understand how the World Bank undertook new actions to support the COVID-19 context. The innovations were identified through a crowdsourcing survey of country teams, review of the project portfolio supporting COVID-19, a review of innovation stories published by Global Practices, and as part of the analysis of the eight case studies conducted for the evaluation.

At the portfolio level:

  • An analysis of operational financing projects and advisory services and analytics supporting the early COVID-19 response was conducted, including analyses of monitoring of the response and procurement. The evaluation undertakes a systematic document and data review focused on internal databases and coded information on a portfolio of COVID-19 projects (appendix B).
  • An analysis of projects under the Multiphase Programmatic Approach led by Health, Nutrition, and Population was conducted. The analysis uses data from the evaluation portfolio, case studies, regional project analysis, and innovation stocktaking to review Multiphase Programmatic Approach projects in the first year of the response—projects approved by April 30, 2021 (appendix H).

At the corporate level:

  • A review of internal processes and partnerships sought to distill lessons and findings on how the World Bank’s COVID-19 internal coordination and collaboration, financing instruments, operational processes, partnerships, knowledge support, digital tools, and monitoring and reporting guided and supported the early COVID-19 response. The review was based on information from document review and individual or group semistructured interviews.

Source: Independent Evaluation Group. The stocktaking of innovations is available in World Bank 2022a.

Limitations

An important limitation of the evaluation is the dynamic nature of the COVID-19 situation and World Bank response, and the overlap of the evaluation with the ongoing response implementation. The World Bank was adapting the response during implementation to improve its actions, given the evolving and uncertain global and country-level contexts of COVID-19. Moreover, interviews were challenging because of the high number of COVID-19 cases in some countries, the burden of the pandemic on health sector personnel, and illness and personal losses of interviewees as a result of COVID-19. Although the evaluation analyses estimate early support and financing in the countries covered by the portfolio, a rapid update of the portfolio was done at the end of the evaluation to gauge shifts in support as the portfolio continued to evolve throughout the evaluation timeline. A future later-stage evaluation is proposed to look at the effectiveness of the response. Appendix A also outlines limitations of specific methods of the evaluation.

Structure

The report structure is based on the theory of action of the evaluation. Each chapter highlights evidence on early support to countries to effectively respond to the health and social shocks of COVID-19 and to start a process to strengthen systems and policy for better crisis preparedness and protection of human capital. Chapter 2 looks at the extensive scale of the World Bank support and its relevance to needs of countries. Chapter 3 covers the implementation successes and challenges of the World Bank’s early support to countries, which can point to early results, and how the World Bank has adapted and learned during implementation. Chapter 4 covers how well operational policies and partnerships facilitated a smooth and speedy response. Chapter 5 synthesizes key areas of learning from the evaluation, which can inform future preparedness, and presents recommendations for the way forward.

  1. An in-depth analysis of COVID-19 commitments and financing allocations is outside the scope of the current evaluation. The evaluation provides an estimate from available data on the portfolio for the time period, countries, and Global Practices covered by the analysis.
  2. The Inform COVID-19 Risk Index was used to categorize countries based on their vulnerability to development achievements being offset by the pandemic. The evaluation adjusted the index to consider the country’s human capital index, given concerns surrounding losses of human capital in countries. The countries were then separated into quartiles based on their vulnerabilities to development and human capital losses (very high vulnerability, high vulnerability, moderate vulnerability, and low vulnerability). Appendix B includes a list of the countries in the portfolio by vulnerability quartile. The Inform COVID-19 Risk Index includes dimensions of social inclusion (such as gender inequality and poverty), economic vulnerability, governance and institutional capacity, health systems capacity, environment, and population risks (such as access to sanitation and population mobility and density; Poljanšek, Vernaccini, and Marin Ferrer 2020; World Bank 2020f).