Recently, I watched (again) the space comedy ‘Mars Attacks!’ which  features a scene where little green men with bulbous heads run around shooting ray guns at everything that moves. All the while, they are carrying a translator box captured from humans that keeps saying “We come in peace!”

As evaluators, we often come across as aliens in our interactions with our counterparts, whose projects we evaluate. I have asked myself often – how did I become the Alien? Did the transformation take place when I was a project manager in a development bank and was asked to prepare a development policy loan in three months based on already implemented policy reforms? When I foundered on the question of whether my efforts mattered or added any value? Or was it after I had watched a whole generation of health loans with identical designs (component 1 – basic health packages delivered by NGOs, component 2 - hospital autonomy, component 3 – health insurance) reformulated and components canceled because they were too complicated?

No, that wasn’t it. In fact, it was when I saw the power of evaluation at play in the highly successful Progresa/Oportunidades Conditional Cash Transfer (CCT) program in Mexico. I realized what a powerful instrument it was to ensure that every dollar spent improved human welfare. It showed us that we needed to target the transfers to ensure transition from primary to secondary school. It showed us that the composition of the nutritional supplements (Galletas) needed to be changed to have the desired effect on anemia among children, because the type of iron used in the formulation of the supplement was not easily absorbed in the body.

Fast forward to my current role at IEG, where we are charged with assessing the World Bank Group’s operations and projects. As part of our job, we ask every day: are we getting the most bang for the buck? After all, a dollar badly spent is a life not saved or improved.

Are we (you) asking the right questions?

Getting the right results should matter for any development practitioner. As a project manager working in development or a task team lead in the World Bank, have you  thought about what you are trying to achieve, and considered how best to achieve this? Granted, the governments or donors have their ideas for what they want their funds to be spent on, but have you had the conversation with them about what they are trying to achieve and presented the latest evidence on what works best to achieve their goal? Have you checked the literature or the experience from other projects and colleagues in your field? Have you informed the authorities about what you know and what you do not yet know about how to achieve meaningful results? These are the questions you should have in mind when you talk about your project’s ‘theory of change’ and the evidence to support it.

According to IEG’s Learning Evaluation, these questions are often not asked by project managers in the World Bank Group, and I would guess the same is probably true for other development agencies and projects. The World Bank Group’s institutional culture tends to place approval and disbursement above impacts, which is driven in large part by misaligned incentives and signals from the leadership. And when IEG assesses whether the project worked (i.e. achieved its objectives), many staff misconstrue this as a judgment on their personal effort and performance (discussed in IEG’s recent ROSES Evaluation). Rather than being interested in learning from results, we find that many Bank Group staff are seemingly obsessed with portraying their projects in a good light and are provoked by IEG’s efforts to disentangle intentions from outcomes, although, really, we come in peace!

When I saw ‘Mars Attacks!’ again, I realized that I had forgotten an important element: the white dove. In an early scene, the movie plays out a tense moment when the Martians step off their space ship and face a large welcome committee of humans. All is peaceful and quiet until the humans release the white dove. Then the violence erupts. In this case, the white dove clearly meant something else in Martian culture than it did to the earthlings!

So here’s a plea to all development practitioners. The next time an evaluator asks you for a clear theory of change and evidence to support it, it is like releasing the white dove that we think everyone should understand. Please don’t shoot it down – that makes YOU the Alien!