The World Bank’s Early Support to Addressing COVID-19
Chapter 3 | Quality of Response: Early Successes, Challenges, Learning, and Adjustment
About half of projects had satisfactory implementation status. About 40 percent of countries had support to monitoring, critical health services, essential health services, and community activities—key for satisfactory implementation and a proxy indicator suggesting that countries are on track for results.
Case studies point to early successes in countries across health, education, and social protection sectors, with continuous efforts to improve targeting of vulnerable groups and better reach frontline workers.
The World Bank made good use of learning from past crises and is implementing intervention types with positive evidence of effectiveness from previous responses, although community activities and the intensity of risk communication were limited.
Innovations in World Bank support to the early COVID-19 response offer an opportunity for systematic learning about how to implement new approaches for crisis preparedness and systems resilience.
Response was swift where World Bank teams and government engaged in useful dialogue and where relationships and national and subnational structures had been developed for coordination and delivery of services before COVID-19.
World Bank teams strongly engaged with governments to make iterative adjustments to improve project implementation. Having real-time data on the quality of crisis-related activities in communities facilitated corrective action; however, data systems capacity in countries was limited.
Regional projects helped countries act rapidly to implement health interventions, but support to regional approaches was limited, despite the readiness and experience of some regional organizations, particularly in Africa. Regional approaches were key for convening, knowledge sharing, and cooperation among government leaders and technical actors implementing responses.
This chapter assesses the quality of the World Bank’s implementation of the early COVID-19 response, including successes and challenges (which can point to early results) and learning to adjust and improve actions in countries. The assessment is based on dimensions of quality from the theory of action in figure 3.1.
Figure 3.1. Dimensions Assessed for Quality of Implementation and Learning
Figure 3.1. Dimensions Assessed for Quality of Implementation and Learning
Source: Independent Evaluation Group portfolio.
Implementation Status and Facilitating Factors
About half of the health and social projects during the early response to COVID-19 had a satisfactory implementation progress rating, with IDA projects reporting better progress than the International Bank for Reconstruction and Development (figure 3.2). At the same time, as could have been expected, the proportion of projects with satisfactory implementation progress is lower than the period before COVID-19, likely as a result of the challenges World Bank teams and countries faced in implementing projects during a pandemic. In countries with more than 40 weeks of community spread of COVID-19 per the WHO classification, more projects have moderately satisfactory or lower implementation progress ratings, suggesting that implementation challenges increase when cases peak.
Figure 3.2. Implementation Progress Ratings of Projects in Countries
Figure 3.2. Implementation Progress Ratings of Projects in Countries
Source: Independent Evaluation Group portfolio.
Note: Projects with no implementation progress rating and projects with financing other than IBRD, IDA, and blend were excluded from the analysis. Implementation Status and Results Report data were from November 5, 2021. The total number of projects is 206. IBRD = International Bank for Reconstruction and Development; IDA = International Development Association.
A mix of interventions in a few key areas, such as monitoring, critical health services (especially for laboratories), essential health services, and community activities, suggest that a World Bank country program is on track to facilitate results for the COVID-19 response. Having a mix of interventions appears to be a factor contributing to satisfactory implementation of the World Bank’s response in countries. The decision tree analysis (appendix D) found that countries with support to monitoring, critical health services (especially laboratories but also in areas of infection prevention and control, case management, and surveillance, where capacities were limited before COVID-19), essential health services such as maternal and child health, and community activities (citizen engagement, gender equality, and urban health, such as communication and sanitation) were more likely to have projects with satisfactory implementation ratings (figure 3.3). About 40 percent of countries had support in most of these areas. Critical health services were well supported, but coverage of citizen engagement, essential health services, gender equality, and urban support was limited (figure 3.4). Interventions in critical health services were especially important in countries less prepared to deliver these services, pointing to the value of supporting countries to prepare for crisis. Case studies and evidence from the literature review (appendixes C and E) also reinforce the finding that essential health services are key for preventing losses of human capital among women and children in a crisis, community activities for health and nutrition messaging, and citizen engagement for trust and communication. A challenge in some case study countries was the limited capacity of the health system to deliver local-level health services, even when there was some crisis preparedness. Addressing gender equality was important in countries where this was a need before COVID-19, pointing to the value of using a gender lens in crisis preparedness and response measures. Providing urban health support within the crisis response was important in countries (such as Haiti, India, Tajikistan, and Uganda) with higher urban risks for the spread of COVID-19 in populations in cities in terms of population density and sanitation, for example.
Figure 3.3. Factors Important to Satisfactory Implementation of Country Support
Figure 3.3. Factors Important to Satisfactory Implementation of Country Support
Source: Independent Evaluation Group portfolio and decision tree analysis.
Note: The size of the leaf corresponds to its importance for predicting the likelihood that projects in that country will be deemed to have satisfactory implementation progress (satisfactory or highly satisfactory): (i) countries with high monitoring and tracking of early evidence of progress (top 50 percent of distribution) were more likely to have projects with satisfactory implementation progress; (ii) countries undertaking even one intervention in these areas were more likely to have projects with satisfactory implementation progress; (iii) in countries with needs in these areas, support to address gender equality and urban health risks was important—having better situations in terms of gender equality (top quartile) and urban health risks (top two quartiles) made it more likely to have projects with satisfactory implementation progress; and (iv) countries with greater preparedness to deliver critical health services (top 50 percent of the distribution) were more likely to have projects with satisfactory implementation status.
Figure 3.4. Coverage of Country Support to Areas Important to Facilitating Satisfactory Implementation
Figure 3.4. Coverage of Country Support to Areas Important to Facilitating Satisfactory Implementation
Source: Independent Evaluation Group portfolio and needs analysis.
Note: Percentages reported for critical health services, gender equality, and urban support measure the extent to which World Bank support was aligned with a country’s needs in those areas. The number of countries reported for each of those three areas (53, 53, and 66 countries, respectively) corresponds to the number of countries with needs in those areas in the bottom two quartiles. For all other areas, the number of countries reported is the number of eligible countries for the evaluation (106). Monitoring reflects countries with any indicators on the COVID-19 response monitored, not the level of monitoring. Urban needs consider urban health risks related to sanitation and water access, household size and type, and population density. Urban support includes health and social activities focused on urban communities. Critical health services include infection prevention and control, case management, surveillance, laboratory support, and risk communication. Needs related to critical health services are based on International Health Regulations data on laboratory, surveillance, and human resource capacities in the country before COVID-19 (appendix D).
Early Results
Although it is too early to observe outcomes, case studies provide some evidence of early outputs, which point to successes of country support (appendix C). Examples of early successes include the expansion of critical health services such as COVID-19 testing, social protection benefits, and remote learning for children (box 3.1).
Box 3.1. Examples of Early Results from Case Study Countries
Ensuring Health Services
In Djibouti, the Multiphase Programmatic Approach (MPA) enabled the development of guidelines and standardized sample collection methods and identified sites for introduction of point-of-care diagnostics. The MPA also helped with supplies for health facilities, such as polymerase chain reaction machines and COVID-19 test kits. By December 2020, the MPA was helping support the investigation of suspected cases of COVID-19 based on national guidelines, in a context of very limited capacity to deliver critical health services. Health workers were also trained in infection prevention and control per nationally approved protocols, and all acute health-care facilities had triage capacity.
Protecting Poor and Vulnerable People
In the Philippines, at the onset of COVID-19, the World Bank helped the government to expand existing cash transfer support to provide monetary and food support to vulnerable households, including people with disabilities and indigenous persons and migrant workers. The support targeted 70 million households, of which 85 percent of recipients were women. The government also fast-tracked the use of digital payment and verification systems and links to the national identification system to improve coverage of vulnerable households and build longer-term capacities for managing emergency assistance. The long-running KALAHI Disaster Response Operations Modality Project was adjusted to provide cash transfers to protect populations during COVID-19, including for employees who lost their jobs and returned to their communities and support for community-run projects focused on building local resilience during COVID-19, such as communal gardens and cleaning of facilities.
Ensuring Child Welfare
In Uganda, the COVID-19 Emergency Education Response Project developed online, paper-based, and radio home-based learning materials for preprimary, primary, and secondary school children and students with social needs, and guided standards and improved sanitation and other conditions in more than 20,000 schools for safe reopening. The project trained more than 10,000 teachers on psychosocial support to counsel learners and school workers on COVID-19 and challenges associated with the lockdown. The World Bank teams also helped the government to develop parenting education and support for early learning continuity, including radio programs.
Risk Communication and Community Engagement
In Senegal, the pandemic meant that community mobilization activities and household visits had to be scaled down to avoid close contact. As soon as the government confirmed the urgency of prevention measures, the Early Years for Human Development Project developed and disseminated guidelines on how to conduct community activities in the context of COVID-19. Communication about preventing the spread of COVID-19 used existing networks, including local radio, to get nutrition information to households. The information also accompanied the provision of food and hygiene kits to high-risk groups, reaching more than 90 percent of targeted populations with messages.
Source: Independent Evaluation Group case study analysis.
Supporting national responses was complemented by adaptive actions during implementation to reach vulnerable groups and support frontline workers. An early success was the focus on vulnerable groups by the Social Protection and Jobs GP and the Macroeconomics, Trade, and Investment GP (more than 90 percent of projects; figure 3.5) and a greater focus on vulnerable groups in FCS countries (appendix B). In Urban, Disaster Risk Management, Resilience, and Land (58 percent of projects); Health, Nutrition, and Population (50 percent of projects); and Education (14 percent of projects), fewer projects targeted vulnerable groups from the onset, though there were efforts to continuously improve support to better reach vulnerable groups. Health and education actions had broad population benefits through rapid expansion of COVID-19 services and online learning. This needed to be complemented with actions to ensure the reach of vulnerable groups, such as women and children, communities with elevated risks of infection, and children in vulnerable households but also frontline workers who were overwhelmed by the crisis. Evidence from the Asian Development Bank, Enabel, the United Nations Children’s Fund (UNICEF), and the International Labour Organization confirms that vulnerable groups needed to be better targeted in the initial design of interventions (Johnson and Kennedy-Chouane 2021; Vancutsem and Mahieu 2020). Case studies also suggest challenges in both health and education in the intensity of support to frontline health workers and teachers to continue to provide services in communities during the crisis. However, in health, about 78 percent of countries had some support (such as to train health workers), and there was large-scale procurement of personal protective equipment across countries for health facilities, which likely benefited frontline workers.
- Health, Nutrition, and Population support offered broad population benefits by financing national plans, but the success of national support in reaching local services was rarely monitored, and there was a need to adapt actions during implementation to ensure the reach of vulnerable local-level groups. For example, in Senegal and Uganda, supporting frontline health workers and communities and vulnerable women and children required the adjustment of actions during implementation.
- Education support was often decentralized (63 percent targeted subnational areas and 19 percent communities; see appendix B) to benefit networks of parents, youth, and children in communities and schools and in some cases, children with special needs. Case studies and the portfolio analysis found that continuous attention was needed to focus actions on the most vulnerable groups, such as children in poor households and girls. Some countries started teacher coaching networks to better support teachers at the front line of the response (such as Djibouti and Uganda).
- Social Protection and Jobs, by expanding systems to migrant workers and female head of households, stands out for its focus on women and girls and vulnerable groups.
- Macroeconomics, Trade, and Investment and Urban, Disaster Risk Management, Resilience, and Land financing had broad population benefits and often supported policies and actions to benefit vulnerable groups, such as farmers, women, informal sector workers, migrant workers, and people in urban slums.
Figure 3.5. Intended Beneficiaries of World Bank Country Support by Global Practice
Figure 3.5. Intended Beneficiaries of World Bank Country Support by Global Practice
Source: Independent Evaluation Group portfolio.
Note: Bars indicate the percentage of projects with at least one World Bank intervention that targeted the beneficiary group. Analysis included 97 countries and 253 coded projects.
Case studies point to some early successes where World Bank support contributed to helping to target vulnerable groups in countries, but challenges remain (box 3.2). The data from available surveys in case study countries suggest some early success with helping countries with risk communication and social protection—key areas to which the World Bank and other partners contributed by, for example, helping to expand social protection responses during COVID-19. Challenges of the early response in terms of reaching vulnerable groups included facilitating access to essential health services (such as for women and children), learning for children, livelihoods of informal workers, and trust and social cohesion.
Box 3.2. Examples of Successes and Challenges of Early COVID-19 Support
Ensuring Health Services
Challenges:
- Surveys in Djibouti and India reported challenges in access to health care. In India, households reported forgoing health care because of fears associated with COVID-19. In Djibouti, Mozambique, Tajikistan, and Uganda, issues with ongoing access to health care were felt more severely by vulnerable groups. In Djibouti and Mozambique, those issues were more acute for women than men.
Protecting Poor and Vulnerable People
Successes:
- Social protection benefits were identified in surveys from Djibouti, India, the Philippines, Senegal, and Tajikistan. For example, India’s social protection response covered a little more than 87 percent of poor households between May and August 2020.
Challenges:
- Negative impacts on the livelihoods of informal workers were reported in Djibouti, India, Mozambique, the Philippines, and Uganda.
- Disparities were reported in Djibouti’s social assistance, with residents outside urban areas less likely to receive food stamps.
Ensuring Child Welfare
Challenges:
- In Honduras, Mozambique, and the Philippines, children reported being unable to access virtual schooling because of issues related to the internet, equipment, and teachers.
- In Djibouti, India, and Mozambique, respondents highlighted negative mental health issues among minority groups. In India and Mozambique, female respondents reported increased mental health issues linked to COVID-19.
- In Uganda, after school reopening, less than half of the children returned to school.
Risk Communication and Community Engagement
Successes:
- In Djibouti, the Philippines, and Tajikistan, high proportions of respondents reported adopting COVID-19 preventive measures, such as social distancing and handwashing.
Challenges:
- In Senegal, disparities in awareness of COVID-19 were reported among women; rural dwellers; and less educated, younger, and poorer populations.
- In Uganda, preventive behaviors declined then stabilized by April 2021, except for handwashing, which continued to decline.
- In Mozambique, Senegal, and Uganda, distrust of government was a reported challenge.
Sources: Afrobarometer 2021a, 2021b, 2021c, 2021d; Bau et al. 2021; Bautista, Balibrea, and Bleza 2020; Bhattacharya and Roy 2021; Grover et al. 2020; Ipsos 2020; Sumalatha, Bhat, and Chitra 2021; Tuppal et al. 2021; WHO 2021b; World Bank 2021c, 2021d, 2021g; World Bank Microdata Library Database (https://microdata.worldbank.org/index.php/catalog/4055; accessed September 2021); UN Women 2021.
Building on Lessons and Evidence from Past Crises
Compared with past crises, the World Bank’s COVID-19 support to countries was stronger in responsiveness to needs; however, some lessons from past crises were not fully integrated. The evaluation analyzed operational lessons from World Bank projects that supported crises over 20 years and benchmarked these against the early COVID-19 response (appendix F) and evaluations of crisis response (World Bank 2012, 2017, 2019a). Close navigation of the response with government is a key implementation success of the COVID-19 response that stands out against past crisis support, especially given the vast scale of the response compared with past crises (table 3.1). However, persistent challenges related to implementation and learning were carried over from past crises in terms of reaching vulnerable groups, multisector coordination in countries, and engaging government in monitoring and using data to set priorities and inform risk communication messaging and behavior change. Success factors in these areas were limited to a few countries. Routine oversight of implementation was challenging because internet connectivity was limited, and it was difficult to communicate with subnational project implementers. These challenges were evident in case studies and in the analysis of success and challenge factors reported in project Implementation Status and Results Reports (figure 3.6).
The World Bank is implementing intervention types with positive evidence of effectiveness from responses to past crises. The evaluation reviewed evidence on effective crisis interventions from systematic reviews and country studies to understand the extent that the current portfolio is positioned to support outcomes in countries (appendix E). The review of evidence identified 70 relevant articles covering 50 interventions relevant to areas of the COVID-19 response framework. Most of the portfolio had interventions with positive evidence of effectiveness from past crises (such as surveillance, case management, infection prevention and control, laboratories, and country-level coordination).
Table 3.1. Application of Operational Lessons from Past Crises in COVID-19
|
Past Lessons to Improve Crisis Response |
Application in Early COVID-19 Response |
Challenges in Early COVID-19 Response |
|
Ongoing responsiveness to needs during crises was supported by frequently engaging with clients to navigate in-the-field realities. |
|
|
|
Coordinating roles and response areas with government and partners throughout implementation was important to address emerging priorities. |
|
|
|
Consistent monitoring of behavior change was important for effective communication approaches. |
|
|
|
Continuous engagement with government helped support corrective actions. |
|
|
|
Engaging government in ongoing monitoring and review helped prioritize support. |
|
|
|
Multisector coordination at national and subnational levels helped ensure an effective response. |
|
|
Source: Independent Evaluation Group lessons analysis and case studies.
Note: Lessons were extracted from 170 closed past projects that supported crisis response. Those shown in the table relate to the ongoing efforts to address needs during implementation and learning, which are compared with actions and challenges of the early COVID-19 response. Refer to appendix F for lessons analysis.
Figure 3.6. Areas of Implementation Successes and Challenges in COVID-19 Response
Figure 3.6. Areas of Implementation Successes and Challenges in COVID-19 Response
Source: Independent Evaluation Group review of project Implementation Status and Results Reports for lessons analysis.
Note: The COVID-19 portfolio includes 132 successes and 149 challenges in the selected areas of the figure, extracted from 119 projects coded with those areas. “Ongoing responsiveness to needs” looks at ongoing support to dialogue, diagnostics, drawing on existing capacities, prioritization of support to sectors, and targeting vulnerable groups. “Coordinating roles and areas of support” looks at the alignment with development actors and plans. “Monitoring of behavior change” looks at communication and the monitoring of barriers and behaviors. “Continuous support for corrective actions” looks at adjustments made through project supervision and management. “Monitoring priorities” looks at use of monitoring data with clients to improve the quality of the response in local areas. “Multisector coordination” looks at coordination across sectors nationally and of subnational actors (appendix F).
Some interventions with positive evidence of effectiveness from past crises, such as risk communication and demand-side activities in communities, were limited in the response. Box 3.3 summarizes interventions with positive and consistent evidence from the systematic reviews and country studies early in the COVID-19 pandemic. Citizen engagement, risk communication, social cohesion, continuation of essential health services, sexual and reproductive health services, and psychosocial support together account for only about 15 percent of the early response portfolio, despite positive evidence of the effectiveness of these approaches. Psychosocial care in communities is an intervention with positive evidence, which may have helped address increasing distress among children (Loperfido et al. 2020). These areas are important to a prevention-oriented response to protect human capital; they also aligned with lessons for crisis response from Ebola and avian influenza (Gold and Hutton 2020; World Bank 2021f). Interventions that have limited evidence in a crisis context offer opportunities for systematic learning, such as remote learning in schools and use of social media.
Box 3.3. Intervention Areas with Positive Evidence and Areas for Learning
Intervention areas with positive evidence:
- Building the capacity of community health workers
- Provision of masks, respirators, and face coverings and infection prevention and control training for health workers
- Combining community prevention measures (masks, hygiene, and physical distancing)
- Telehealth for continuation of essential health services
- Providing sexual and reproductive health services in emergencies
- Surging capacity of human resources and adaptation of health facilities for case management
- Active case surveillance and contact tracing, combined with rapid diagnosis and management, and quarantine measures
- Strengthening health information and surveillance systems
- Community engagement for risk communication, infection prevention and control, hand hygiene, use of masks, and social distancing
- Combining prevention communication with community-based messaging
- Engaging existing community leaders and community-based structures
- Mental health and psychosocial support programs in community and health structures
- Unconditional cash transfers for social protection
- Financial and social support for protection of vulnerable girls in humanitarian settings
- Point-of-care diagnostics, rationing medical supplies
- Support for prompt and consistent policies in epidemics and regional coordination
Intervention areas where evidence is inconsistent or lacking:
- Digital and automated tools for case management and surveillance
- Health workers’ use of other personal protective equipment (gloves, gowns, and eyewear)
- Community-based surveillance
- Social media for risk communication and monitoring of response and needs
- Remote learning and school reopening measures for vulnerable populations
- Models for supporting logistics and medical supply in crisis
- Workplace mental health
Source: Independent Evaluation Group portfolio.
Innovation and Learning
Innovations to which the World Bank’s financing and technical support to the early COVID-19 response contributed offer important learning that could be used to build more resilient systems for recovery. Stocktaking analysis identified innovations in more than 80 percent of countries in the portfolio, often reflecting new approaches or practices to strengthen systems (World Bank 2022a). Innovations to support the response were positively associated with the reorientation of World Bank country portfolios, suggesting that reorientation opens opportunities for innovation. Innovations were also encouraged through regional projects, often building on experiences and investments before COVID-19. Assessing the effectiveness of these innovations to understand the benefit that they provided in the country was outside the scope of the evaluation but will be important. Monitoring the quality of health services and expanding delivery (such as through telehealth), expanding social registries using data analytics to identify vulnerable groups, and public-private partnerships to expand digitalization of systems were common areas of innovation. The engagement of local actors, multiple sectors (such as water, technology, and social sectors), and partnerships was observed across innovations (box 3.4). Innovations and ASA often addressed areas where evidence-based learning is important, such as remote learning support for schools. Innovations also often involved civil society and the private sector and point to opportunities to expand engagement of these groups in recovery to strengthen preparedness for crisis response.
Further learning could help World Bank teams address areas that are important but received limited attention in the early response. Few innovations (less than 5 percent) were found to support continuation of essential health services, the informal sector, risk communication, psychosocial support, citizen engagement, and social cohesion, which are limited in the World Bank’s early COVID-19 response. Moreover, only 10 percent of innovations identified in the COVID-19 response addressed gender disparities (World Bank 2022a). Evidence from the literature and past lessons also emphasize the importance of these areas for crisis response (Gold and Hutton 2020; World Bank 2021f). Support to manage burnout among local health-care workers, handle misinformation, and garner trust, which case studies and interviews identified as challenges, received very little attention.
Learning about the effectiveness of interventions, especially where evidence is inconsistent or lacking, could help ensure the right mix of support to countries. Remote learning support in crises stands out as a widespread intervention undertaken across countries for which the rapid review conducted by the evaluation did not identify evidence, although there may be evidence from noncrisis contexts that is transferable. Other interventions for which evidence of effectiveness is inconsistent or lacking include the use of social media in crises to communicate messages, the use of digital and automated tools for case management, and logistics supply management. Community-level surveillance has also been important in countries where it has been supported. Deepening of evidence could also be helpful in areas with few studies, such as for the expansion of social protection in emergencies. For example, evidence was collected on social protection interventions during the early COVID-19 response (Gentilini 2022).
Box 3.4. Examples of Innovations Supporting the COVID-19 Response
- In Uzbekistan, people receive information about COVID-19 through SMS messages, Telegram, WhatsApp, video clips, and infographics. In addition, health services are adapting for telemedicine where possible.
- In Mali, a new national 24-hour-a-day, 7-day-a-week call center dedicated to COVID-19 enables free calls and offers advice for implementing coronavirus protocols.
- In Senegal, community-based disease surveillance and multistakeholder engagement allow community health workers and volunteers to detect COVID-19 and report cases to health facilities and local government agencies.
- In Latin America, projects track the presence of COVID-19 in wastewater. Wastewater-based epidemiology supplies real-time information on the extent of virus spread in a community, including asymptomatic cases. Each sample represents a large portion of the community, which is served by a sewerage network; this allows for rapid and cost-effective tracking of disease trends at the population level.
- In Cambodia and India, instructional videos, conference calls, and social media supplement coaching services for teachers. Rural teachers receive video lessons on teaching culturally relevant, curricula-aligned content.
Source: Independent Evaluation Group innovation stocktaking.
Note: SMS = short messaging service.
Coordination, Dialogue, and Adjustment in Countries
Good pre–COVID-19 relationships were a factor for good dialogue and smoother implementation. Previous evaluations related to financial, social, and environmental shocks also establish the importance of country dialogue in crisis response (World Bank 2012, 2017). Commitment to dialogue with government was strong, often with weekly discussions of emerging challenges and urgent reforms. Good relationships before COVID-19 prepared countries for strong implementation. Examples include the following:
- In Honduras, the country’s social response built on long-standing policy dialogue with the government, which deepened through daily exchanges and was informed by ASA. In health, a new policy dialogue developed about emergency measures, which initially slowed the response but later transitioned to a dialogue aimed at ensuring better health systems. In education, the dialogue helped with quick project adjustments.
- In the Philippines, the MPA supported the first health project after years of no project. Hence, health dialogue with the World Bank deepened in the early COVID-19 response and then expanded to develop early access to vaccines and planning of health systems strengthening.
- In Mozambique, without a centrally organized government response, the World Bank led sector-specific responses in dialogue with relevant ministries and other development partners. Support focused on the health, education, urban, and social protection sectors. The response built on existing sector relationships and accelerated the pace and direction of measures underway before the pandemic.
Where available, national platforms helped engage government sectors and development partners to coordinate implementation. Case studies show good engagement in national coordinating structures of government during implementation, even though multisector coordination structures were rare (box 3.5). For example, in India, the Philippines, and Uganda, the World Bank supported coordination platforms during COVID-19. Some World Bank teams helped reinforce government’s coordination capacity, although it was challenging to reinforce amid the crisis. For example, Tajikistan hired a consultant to help with coordination. In Uganda, a dedicated staff member in the World Bank office supported partner coordination with government. Moreover, an important weakness of the World Bank’s support to the response was the limited engagement with nongovernmental organizations—an issue identified with many COVID-19 responses (OECD 2022).
Box 3.5. Lessons on Multisector Coordination for an Integrated Response
Where countries had established coordination structures to engage government sectors, partners, and other stakeholders, the World Bank supported them, and the government used them to plan and track integrated COVID-19 actions. About 10 percent of countries had support to multisector coordination groups to implement the response, such as One Health committees. Having these structures set up before COVID-19 to organize the response was key because setting up coordination for the first time is challenging. Having multisector coordination structures was also important for crisis preparedness. The World Bank’s COVID-19 recovery efforts are emphasizing the establishment of One Health coordination within countries to support multisector responses and strengthen coordination structures. This aligns with the efforts of other agencies—the Food and Agriculture Organization, the World Organisation for Animal Health, the United Nations Environment Programme, and the World Health Organization.
When multisector structures were developed, they provided a platform for countries to plan, report on, and take rapid actions with more integration across sectors:
- In Haiti, the government created a multisectoral commission to coordinate the COVID-19 response, integrating mechanisms for civil society and the private sector to contribute to emergency preparedness actions, and specific units for crisis response in the health sector. This helped align sector and stakeholder support to the response.
- In the Philippines, the national response was coordinated by an intersectoral task force initially led by the health sector, but early in the response, the task force transitioned to central government leadership to ensure multisector support across ministries.
- In Senegal, the World Bank supported a One Health multisectoral approach to coordinate the COVID-19 response. This approach grew to include ministries responsible for finance, health, social affairs, livestock and animals, agriculture, rural development, environment and sustainable development, and water and sanitation. Since COVID-19, the approach has included education.
Knowledge work in COVID-19 reinforced coordination by helping inform how to operationalize crisis response actions. Previous evaluations also found that coordinating support with government and partners, combined with knowledge work, enabled the World Bank to develop well-designed financing projects expeditiously (World Bank 2017). For example, the multisectoral response in the Philippines built on long-term knowledge work in social protection and community development.
Sources: Independent Evaluation Group portfolio and case studies; FAO et al. 2022.
Where available, networks that reached communities were instrumental in risk communication, detecting COVID-19 cases, and providing referrals to health services, but overall, the connection to local government—especially at the community level—was not strong enough. Enabling local government and community groups to support crisis response and ensure frontline service delivery was a challenge in case study countries, with breaks in communication, coordination, and disruptions in implementation. Evaluations of the responses by Chazaly and Goldman (2021) and the German Institute for Development Evaluation (Schneider et al. 2020) also noted similar issues. Examples of coordination support reaching local levels to a greater degree include the following:
- Structures for nutrition (Honduras, Senegal, and Uganda) facilitated COVID-19 messaging to communities and helped engage nongovernmental organizations in the COVID-19 response. The World Bank is undertaking similar work in developing community networks in other countries (Subandoro, Holschneider, and Ruel-Bergeron 2021).
- One Health platforms with networks that reach the community level (Senegal and Zambia) supported contact tracing, case management, communication, and other activities to prevent the spread of disease.
- Support to local government (Senegal and Uganda) helped ensure continued delivery of local services, such as health, water, and education.
Implementation Adjustment
Throughout the early response, World Bank teams strongly engaged with governments to make iterative adjustments to improve implementation. These adjustments were informed by frequent meetings and virtual supervision support, findings from studies, and field supervision by the governments when travel was possible. Discussions often identified implementation bottlenecks and led to corrective actions to adjust projects, such as the following:
- In Djibouti, the package of services delivered to vulnerable groups by social protection agents was adapted to include COVID-19 messages and distribution of hygiene kits, based on a challenge identified in country discussions.
- In Senegal, the health project added support to help the government recruit contractual staff to surge resources for patient care and provide computer and video equipment for communication between health offices.
- In Tajikistan, the country strengthened its support to community engagement and risk communication as part of its vaccine response based on learning from the early response.
- In Uganda, the country added COVID-19 training support for the private sector based on challenges in support to these facilities.
Real-Time Data for Decision-Making
Real-time data, where available, were key to informing decision-making about project adjustments. Geo-enabled monitoring, iterative beneficiary monitoring, short messaging service (SMS) texts, online surveys, and dashboards are tools that supplied timely data to support implementation decisions in the early COVID-19 response (box 3.6). Global surveys provided valuable real-time information on the socioeconomic impact of COVID-19, which informed policy dialogue and country economic updates and were critical for discussion with civil society forums in countries. However, these surveys often did not offer real-time information to support immediate course corrections of projects or develop data capacities in countries. In some countries, real-time surveys were adapted to better link to social protection, vaccine promotion, and education projects to inform course corrections. Evaluations of COVID-19 responses have also found the application of real-time data collection methods to be useful in adapting projects (Johnson and Kennedy-Chouane 2021; OECD 2022; Vancutsem and Mahieu 2020), and a range of innovative data tools have been used in COVID-19. A challenge is supporting governments to continue real-time data methods that enhance country-level information systems and to coordinate data collection that ensures that the most helpful information for implementation decisions is being collected.
Box 3.6. Examples of Real-Time Data Systems and Tools for Decision-Making
Monitoring of country situations:
- In Cambodia, Myanmar, and other countries with high mobile phone coverage, high-frequency phone surveys provided rapid, real-time data and evidence on the socioeconomic impact of COVID-19 to inform World Bank responses.
- In Colombia, a COVID-19 Safe Economic Reactivation Dashboard provides decision makers with real-time information for 1,100 municipalities on key epidemiological indicators. To date, the dashboard has more than 15,000 unique users.
- In Fiji, the World Bank supported the Ministry of Health and Medical Services to improve communication and data reporting systems with frontline health workers and health facilities, including internet connectivity for case reporting and public health surveillance across health facilities.
- In Iraq, Lebanon, Libya, Tunisia, and West Bank and Gaza, a low-cost, just-in-time survey was deployed via a Facebook chatbot to understand attitudes toward vaccines among different social groups. To ensure a degree of national representativeness, the survey targeted clusters of individuals according to region, age, and gender.
Monitoring of implementation quality:
- In Lesotho, a phone-based application sends an SMS message to those who are vaccinated to collect information on the quality of care and track minor side effects that would otherwise not be reported.
- In Tunisia, COVID-19 cash transfers are remotely monitored using iterative beneficiary monitoring.
- In Mexico, Nicaragua, and other countries, the Geo-Enabling Monitoring and Supervision (GEMS) initiative has been introduced as a project supervision and monitoring tool for continuous engagement with communities and project supervision. The governments undertook the capacity-building program on GEMS and internalized the technology tools for monitoring. For example, in Uganda, GEMS is used for real-time project monitoring of support to schools and community groups to promote nutrition. In Mali, GEMS is a platform for third-party monitoring in conflict-affected remote areas. Tajikistan is using GEMS to monitor cash transfers.
- In India, the Gujrat Command and Control Centre provides an example of real-time performance data for schools, covering online attendance, assessment test results, and a vehicle tracking system.
Sources: Independent Evaluation Group portfolio and case studies; World Bank 2022a.
Note: SMS = short messaging service.
Data on implementation quality in subnational and community responses by local governments remain a key gap in the early COVID-19 response. Few countries had systems established before COVID-19 that could be used to provide beneficiary feedback on services or data on the quality and coverage of services delivered by health workers, schools, and community actors, especially for vulnerable groups. Another challenge in the response was tracking the success of messaging and communication activities in promoting behavior changes in the community related to COVID-19. Strengthening country systems to supply more real-time information on the implementation quality of local government and community-level support could facilitate preparedness efforts.
Regional Knowledge Sharing and Cooperation
Regional projects strongly supported implementation of responses. The evaluation reviewed the four main regional disease-focused projects that worked on the COVID-19 crisis in countries with weak health systems or limited capacities to respond to crises, highlighting strong early results (box 3.7 and appendix G). Coordination facilitated by these regional projects supported political leadership; real-time technical learning to operationalize and review progress of COVID-19 plans; cooperation for efficiencies across countries, such as for procurement of medical goods and equipment; joint training to expand COVID-19 surveillance data and testing; and high-level dialogue to develop public health guidelines. The peer learning of ministries and technical experts across countries that engaged in regional activities helped expand leadership for the response and had a spillover effect on countries not covered by regional projects that joined the activities. Leaders and technical experts from public health institutions engaged in regional exchanges and then adopted new practices learned from these engagements to expand COVID-19 actions in their countries. Despite the strong positive early results of regional projects during the COVID-19 response, support to regional projects was limited to only 23 percent of countries in the portfolio, although there has since been some effort to expand regional support in Africa in the second year of the response. A previous IEG evaluation highlighted the untapped potential in fostering regional integration initiatives as the World Bank can leverage its global knowledge, financing instruments, synergies from acting as one Bank Group, and its ability to catalyze regional actors (World Bank 2019c).
Box 3.7. Examples of Early Results Contributed by Regional Projects
The evaluation conducted interviews with country actors and teams involved in regional projects. Among the four projects reviewed, Regional Disease Surveillance Systems Enhancement Project in West Africa and East Africa Public Health Laboratory Network had established networks in countries before the COVID-19 pandemic started, whereas the Africa Centres for Disease Control and Prevention and Organisation of Eastern Caribbean States regional health projects were at early stages of implementation. For all projects, the knowledge exchange and real-time dialogue and coordination facilitated by regional engagement was viewed consistently as valuable to help support country actions.
Early results were identified for all four regional projects, although there was more evidence in countries with a longer duration of regional support before COVID-19 (such as Senegal and Togo, covered by early phases of the Regional Disease Surveillance Systems Enhancement Project). Regional projects were especially well-situated to support critical health services for the COVID-19 response when projects were already established and well-integrated with project support in their respective countries. Early results supported by regional projects include the following:
- Regional coordination facilitated rapid country responses to COVID-19, whether of ministerial committees, public health institutes, or project leaders. Coordination mechanisms were used to share real-time information and knowledge on disease detection and best practices, engage with regional and international partners, develop guidance on surveillance, and exchange knowledge with peers to help countries implement COVID-19 support. For example, governments in the Eastern Caribbean used regional knowledge sharing about health waste management and installation of laboratory and health equipment to respond faster. In Africa, public health institutes and health ministries used the Africa Centres for Disease Control and Prevention regional Extension for Community Healthcare Outcomes platforms for interdisciplinary knowledge exchange and training on COVID-19.
- Human resources capacities supported by regional projects helped implement the COVID-19 response. For example, countries in West Africa and the Eastern Caribbean deployed field epidemiology graduates in leading strategic and front-line roles for COVID-19. Capacities developed before COVID-19 were crucial to rapid responses in East and West Africa. In East Africa, governments deployed lab technicians, assessors, and disease surveillance officers trained through East Africa Public Health Laboratory Network to be at the front line in COVID-19 rapid response teams, conducting testing and contact tracing. Countries used rapid response teams supported by the Regional Disease Surveillance Systems Enhancement Project to implement the COVID-19 response.
- Regional coordination and capacity building facilitated the expansion of surveillance, testing, border screening, case management, and infection prevention and control for the COVID-19 response. For example, East Africa countries deployed laboratory capacity built by the East Africa Public Health Laboratory Network project for COVID-19 testing. The Africa Centres for Disease Control and Prevention contributed to the rollout and expansion of COVID-19 testing in Africa. At the start of COVID-19, only two laboratories in Africa (Senegal and South Africa) could reliably test for the disease. By mid-March 2020, 43 countries had testing capability, and by August, almost all African Union countries could conduct testing.
Source: Independent Evaluation Group regional project review.
Regional projects were well situated to support countries in efficiently cooperating to expand critical health services but were less prepared to help countries plan actions to mitigate COVID-19 impacts. Regional projects helped countries expand testing, surveillance, and case management quickly. They were less helpful in sharing knowledge to implement interventions for risk communication, citizen engagement, gender equality, urban public health risks, and essential services. Discussion is ongoing on expanding attention to these areas in regional disease-focused projects, based on learning from the early COVID-19 response. Capacities could be developed across countries, such as for citizen engagement, through learning to facilitate actions. Improving essential health services in countries may also be important to address regionally because having this capacity helped countries respond quicker.
Strengthening developing capacities of regional organizations for disease response coordination proved to be important. Institutional strengthening before COVID-19 prepared regional organizations to support results during COVID-19, particularly the Economic Community of West African States. Setting up dialogue across countries for the first time during a crisis was more difficult for newer regional projects because actors have limited experience coordinating support and need time to set expectations and develop partnerships and trust. Nevertheless, new platforms (such as the Regional Coordination Center in Zambia for Southern Africa, supported by the Africa Centres for Disease Control and Prevention [Africa CDC] project) were important for learning and cooperation during COVID-19. The Caribbean Public Health Agency, supported by the Organisation of Eastern Caribbean States Regional Health Project, played an important coordination role in facilitating country responses during COVID-19.
The World Bank did not strengthen support to regional approaches early in COVID-19. In Africa, expanding regional support at the onset of COVID-19 may have facilitated the response, such as through including regional projects in the MPA and by providing financing to regional organizations for knowledge sharing and coordination across countries. The Regional Disease Surveillance Systems Enhancement Project in West Africa and the East Africa Public Health Laboratory Network project already had extensive experience before COVID-19 in demonstrating learning about how to work regionally with countries on disease preparedness and response. Moreover, before COVID-19, there was already extensive preparatory work on the Africa CDC project; although the project was at an early stage of implementation and had limited disbursement, much important work to develop coordination and knowledge sharing structures was already in place and could have been drawn on by the World Bank to facilitate the early actions in countries.
Hands-on technical support and facilitating learning, convening, and cooperation across countries were important for regional capacity building. Developing the capacity of regional organizations to facilitate high-level country exchanges among leaders and technical learning and cooperation was critical. Project financing to implement interventions was often from a country-level project, and minimum regional project financing and disbursement often were required to influence the leadership and implementation improvements observed through regional engagement. For example, regional support of the Africa CDC was linked to financing of World Bank projects in Ethiopia and Zambia. Expanded regional support could help develop leadership and coordination in countries, which could then help scale up preparedness capacities and improve the efficient use of resources in World Bank country projects. Learning across countries could be supported through regional projects or ASA focused on regional capacity building and South-South knowledge sharing (World Bank 2019b).