We need to develop methods to assess whether individual interventions produce more together than on their own. That is necessary already in the current context when focusing on country programs, but will be needed even more in cases of "aid-trade-diplomacy" or for the SDGs where some are mutually reinforcing and others will require tough trade offs.

TWEET THIS

• Evaluating policy coherence will require us to expand our vocabulary and evaluation tools.
• We need to develop methods to assess whether individual interventions produce more together than on their own, especially when evaluating policy coherence, or for the SDGs, which can be mutually reinforcing or require tough trade offs.

In my last blog, I shared some takeaways from a meeting I attended recently convened by OECD’s Development Assistance Committee (DAC) – a group that brings together some of the largest providers of international development assistance.

At the same meeting, representatives from other evaluation offices spoke about challenges and opportunities related to the merger of development agencies with departments of foreign affairs and of trade. At the time when countries started merging these ministries several years ago, I remember conversations that expressed concerns about the potential impact. Would aid lose its altruistic nature? Would it become an overt tool of foreign policy, or aim to generate trade benefits for the donor country? Refreshingly, the discussion recognized that this link had always existed, but was now more transparent.

I was particularly impressed by the example of New Zealand. In an insightful presentation, Ingrid van Aalst, Manager, Evaluation and Research at the New Zealand Ministry of Foreign Affairs and Trade, talked about her country’s experience integrating its Ministries of Foreign Affairs and Trade. She acknowledged the inherent difficulties of evaluating across the three dimensions – development, diplomacy and trade. A challenge that was compounded by the fact that in most countries, evaluation has focused on development. Meaning: monies coming from the donor, but actions taking place in another, the recipient country. Generally speaking, ministries other than those having to do with development, have much less experience with evaluation, and less of a culture to do so. While meeting with the usual challenges, it is an opportunity to share the wisdom of evaluative thinking more broadly. Something that will be particularly important in the age of universal SDGs, which all countries are expected to meet.

Likewise, our evaluation colleagues from New Zealand took the challenge as an opportunity to use evaluation strategically. They stepped up to reviewing country relationships from three angles: aid, trade, and peace and security. This approach has allowed them to review policy coherence across these three dimensions to show areas of misalignment. More importantly, the evaluation process is helping clarify New Zealand's vision for working with the country in question, and strategically aligning all three departments.

Bob Picciotto, one of my predecessors, has for many years called for such an assessment of policy coherence, even beyond these three areas of aid, trade, and diplomacy. I agree with him on the need to assess policy coherence, but also I see gaps in the evaluation landscape and in our work at IEG. For example, IEG evaluates what the World Bank Group does in client countries, and by extension how the client countries engage in these programs. But, IEG does not have a mandate to assess policy coherence in the way donor countries engage with developing countries or international development agencies, such as the World Bank Group and UN.

So, if this is a predominantly bilateral affair, how does it matter to evaluation in the multilateral context? In several ways:

Evaluating policy coherence will require us to expand our vocabulary and evaluation tools. Years ago, coherence was introduced by evaluators in the humanitarian sector, and later absorbed into the DAC criteria. But, coherence as an evaluation criterion did not gain as much currency as the other five well recognized ones.

In addition, as I have argued before, we need to develop methods to assess whether individual interventions produce more together than on their own. That is necessary already in the current context when focusing on country programs, but will be needed even more in cases of "aid-trade-diplomacy" or for the SDGs where some are mutually reinforcing and others will require tough trade offs.

These two concepts—coherence and synergy—complement each other, but are different. Policies might be coherent in that they align and do not contradict each other. But, that does not necessarily demonstrate whether they produced greater value added than if they had taken place in isolation. Coherence is probably a necessary condition for synergy, but not a sufficient one.

By adding these criteria to our repertoire, evaluation can focus the attention of policy makers, program designers and implementers on these important results.

Read more from the series Rethinking Evaluation.