Bowling in the dark: Monitoring and evaluation during COVID-19 (Coronavirus)
Lessons from past experience can help creatively and responsibly adapt M&E practices.
As the implications of the Covid-19 pandemic are felt around the world, few will escape its economic repercussions. Competition for financial resources will be fierce, not only between developing economies but between developing and industrial economies. Bolstering the ability of governments to maintain domestic support for global efforts to respond to the needs of the most vulnerable in the midst of this crisis will be critical.
Institutions like the World Bank Group (WBG), as they seek to respond to COVID-19 impacts, as well as development needs more broadly, will need to assure taxpayers and voters in donor countries that increasingly scarce resources are being well spent. This will require creativity in adapting monitoring and evaluation (M&E) systems to match both the speed and forms of the World Bank Group’s crisis response and its continuing pursuit of its development mandate.
World Bank teams with experience working in “no-go” areas with high risks to personal security have much to teach about M&E in the face of COVID-19.
In situations of conflict, one of the ways in which M&E (and data collection in particular) has been carried out has been by drawing on Third-Party Monitoring. However as discussed by Lauren Kelly and Marie Gaarder in a previous IEG blog, there are ethical considerations associated with sending people into harm’s way. These have been exacerbated by COVID, and must be taken into account in M&E efforts. This is because there is significant risk of third-party monitors both spreading and contracting the disease, including between densely populated capitals and more remote, yet unaffected areas. These risks are further compounded by inadequate supplies of protective equipment and a lack of access to COVID-19 testing in many of the countries in which the World Bank operates.
To address these risks, Elias Sagmeister of the Global Public Policy Institute provides a useful toolkit of tech-enabled handheld devices for digital data collection, including mobile phone-based feedback mechanisms, remote sensing with satellites or delivery tracking, communication with online platforms and broadcasting with radios and other forms of (social) media. But technology has its limitations.
If there is one thing we have learned from Ebola, it is that communication matters. In the times of COVID-19, like with Ebola, there will be heightened fear, and confusion, and distrust of governments and donors who are “trying to do the right thing” by conveying hard messages about health and behavior to isolated, disbelieving communities.
M&E has a critical role to play during a pandemic in assessing the continued appropriateness of World Bank activities given the enhanced need to protect personal safety. Do current projects require citizens to participate in-person, in unsafe ways to achieve intended benefits? Are projects helping clients to adapt these systems – education, health services, labor, payments - to maintain essential services and income - while protecting citizen safety? And how will we know?
From a communications standpoint, one of the key lessons from West Africa is that responders’ unfamiliarity with a local culture can undermine pandemic response. “Foreigners working for the UN, the Red Cross and other international organizations were not necessarily the most effective communicators on Ebola.” This is because when people’s lives are threatened, they want to hear from those closest to them—those who speak their language. M&E has a role to play in helping people and projects to adjust modalities alongside the COVID realities. To achieve this, both tech-enabled and human data collection and evaluation efforts need to be culturally and linguistically sensitive.
Countries providing the essential resources for the pandemic response will want to know how funds released through a project’s “Contingency Emergency Response Components” are being spent, but project M&E frameworks are often not equipped to do this. Restructured, adaptive M&E and safe data gathering methods need to be developed alongside the release of such funds, in real time.
This will be particularly important as the global community commits vast sums to support developing countries in responding to COVID-19, including the recently approved $14 billion package of fast-track financing initiative from the World Bank Group to assist companies and countries in their efforts to prevent, detect and respond to the rapid spread of COVID-19. The challenge is to ensure that M&E is not forgotten in the well intentioned rush to respond to urgent needs.
Do you have ideas, lessons, or insights to contribute to this conversation about evaluation during COVID-19? Please share them in the comments below.