As the implications of the Covid-19 pandemic are felt around the world, few will escape its economic repercussions.  Competition for financial resources will be fierce, not only between developing economies but between developing and industrial economies. Bolstering the  ability of governments to maintain domestic support for global efforts to respond to the needs of the most vulnerable in the midst of this crisis will be critical.

Institutions like the World Bank Group (WBG), as they seek to respond to COVID-19 impacts, as well as development needs more broadly, will need to assure taxpayers and voters in donor countries that increasingly scarce resources are being well spent.  This will require creativity in adapting monitoring and evaluation (M&E) systems to match both the speed and forms of the World Bank Group’s crisis response and its continuing pursuit of its development mandate.

Applying Lessons from M&E in Insecure Areas to COVID-19

World Bank teams with experience working in “no-go” areas with high risks to personal security have much to teach about M&E in the face of COVID-19.

In situations of conflict, one of the ways in which M&E (and data collection in particular) has been carried out has been by drawing on Third-Party Monitoring. However as discussed by Lauren Kelly and Marie Gaarder in a previous IEG blog, there are ethical considerations associated with sending people into harm’s way.  These have been  exacerbated  by COVID, and must be taken into account in M&E efforts.  This is because there is significant risk of third-party monitors both spreading and contracting the disease, including between densely populated capitals and more remote, yet unaffected areas. These risks are further compounded by inadequate supplies of protective equipment and a lack of access to COVID-19 testing in many of the countries in which the World Bank operates. 

To address these risks, Elias Sagmeister of the Global Public Policy Institute provides a useful toolkit of tech-enabled handheld devices for digital data collection, including mobile phone-based feedback mechanisms, remote sensing with satellites or delivery tracking, communication with online platforms and broadcasting with radios and other forms of (social) media.  But technology has its limitations.

• First, technology tools need to be designed in line with data collection requirements. Many tech enabled tools are best used to verify assets; they are less equipped to measure quality and implementation fidelity.
• Second, they work best as “a bundle”: earth observation technologies paired with phone-based mechanisms can help verify that assets were delivered and to where, as well as to whom, they were intended to be delivered.
• Third, one needs to be mindful of exclusionary factors surrounding the use of technology. In insecure areas, women often have less access to cell phones than men and this can bias reported outcomes, as was shown in IEG’s radio and mobile-based outreach in the Afghanistan Country Program Evaluation.
• Fourth, data collected by tech enabled tools are unlikely to be able to capture the granularity needed to ascertain and address critical environmental, social and conflict related risks, that are more adequately identified through consultations, rigorous supervision, and implementation of citizen engagement processes.

Lessons Also Can be Gleaned from the Ebola Experience

If there is one thing we have learned from Ebola, it is that communication matters. In the times of COVID-19, like with Ebola, there will be heightened fear, and confusion, and distrust of governments and donors who are “trying to do the right thing” by conveying hard messages about health and behavior to isolated, disbelieving communities.  

M&E has a critical role to play during a pandemic in assessing the continued appropriateness of World Bank activities given the enhanced need to protect personal safety. Do current projects require citizens to participate  in-person, in unsafe ways to achieve intended benefits?  Are projects helping clients to adapt these systems – education, health services, labor,  payments -  to maintain essential services and income - while protecting citizen safety? And how will we know?

From a communications standpoint, one of the key lessons from West Africa is that responders’ unfamiliarity with a local culture can undermine pandemic response.  “Foreigners working for the UN, the Red Cross and other international organizations were not necessarily the most effective communicators on Ebola.” This is because when people’s lives are threatened, they want to hear from those closest to them—those who speak their language.  M&E has a role to play in helping people and projects to adjust modalities alongside the COVID realities. To achieve this, both tech-enabled and human data collection and evaluation efforts need to be culturally and linguistically  sensitive.   

Countries providing the essential resources for the pandemic response will want to know how funds released through a project’s “Contingency Emergency Response Components” are being spent, but project M&E frameworks are often not equipped to do this. Restructured, adaptive M&E and safe data gathering methods need to be developed alongside the release of such  funds, in real time. 

This will be particularly important as the global community commits vast sums to support developing countries in responding to COVID-19, including the recently approved $14 billion package of fast-track financing initiative from the World Bank Group to assist companies and countries in their efforts to prevent, detect and respond to the rapid spread of COVID-19.  The challenge is to ensure that M&E is not forgotten in the well intentioned rush to respond to urgent needs.

Do you have ideas, lessons, or insights to contribute to this conversation about evaluation during COVID-19?  Please share them in the comments below.

photo credit: Shutterstock/ Ms.Lotus Bua