I am a strong believer in the need for and future of evaluation systems within countries. More and more middle- and low-income countries are interested in and committed to investing in evaluation systems. They increasingly see the value of evaluation evidence that will help them make better informed decisions. Developing evaluation capacity to assess how public policies and programs stimulate progress towards the SDGs will provide the opportunity to evaluate policy coherence as well.

Late last year, I participated in a thought-provoking conference on the Sustainable Development Goals (SDGs), partnerships, and evaluation at Wilton Park. It was a great meeting of diverse minds from many different backgrounds.

Policy coherence came up, not as a new issue but one of continued importance. Years ago, evaluators like Bob Picciotto called for evaluations to focus not just on aid policies, but domestic policies that can have a stronger impact on development outcomes than aid.

Today, this question is even more central to the development discourse for a number of reasons. Official Development Assistance as a proportion of total resources – domestic public finance and private sector flows – has shrunk. The SDGs span a much larger number of sectors than their predecessors. The Goals themselves, while centered on specific sectors or issues, are sometimes mutually reinforcing, at other times in competition with each other. Above all, they demonstrate how countries around the world are interdependent. The complexity of development processes highlight the importance of policy coherence while at the same time making it harder to achieve just that. For its part, evaluation will need to draw on methods to be responsive to complexity, as discussed by Bamberger et al in their book “Dealing with Complexity in Development Evaluation”.

Invariably when I am at conferences like these, IEG is looked towards as a leader. The World Bank Group covers many sectors, hence reviewing policy coherence among the many sectors and interventions should be possible.

Yes it is (and is done to certain extent), but the question I am asked focuses on policy coherence of high-income countries. Do their aid policies align with their domestic policies, or their policies for international trade? Or, as Rob van den Berg, former director of the independent evaluation office at the Global Environmental Facility put it: a donor might contribute a considerable amount towards climate issues, but pursue domestic policies that have a much larger impact on the climate agenda than the contribution can make up for.

I agree that these questions need evaluating, but also recognize the institutional limitations of typical evaluation offices. For instance, it is IEG’s role to assess the development effectiveness of the World Bank Group, but not of each or all of its shareholders. What we can do is understand better what has influenced the changes we observe in client countries, and look beyond World Bank Group interventions to explain how these changes came about. It takes moving away from a strict focus on interventions to one that understands the complex set of factors that affect development results. 

In addition, I am a strong believer in the need for and future of evaluation systems within countries. More and more middle- and low-income countries are interested in and committed to investing in evaluation systems. They increasingly see the value of evaluation evidence that will help them make better informed decisions. Developing evaluation capacity to assess how public policies and programs stimulate progress towards the SDGs will provide the opportunity to evaluate policy coherence as well. These evaluation capacities should be built to assess the gamut of (and coherence among) public policy and programs, funded domestically through revenues or (foreign) borrowing, grant-funded programs, and private sector investments.

This is a big ask. Even the leaders among middle-income countries have not built systems as comprehensive as this. Take the example of Mexico. CONEVAL is a highly respected and powerful independent evaluation institution in Mexico. It evaluates the effectiveness of Mexico’s social development policies, and has made impressive inroads into closing the feedback loop to ensure policy-makers understand and act on evaluation evidence. Social development policies play an important part in fighting poverty. But, a focus on (full-fledged) policy coherence would ensure that evaluation assesses also economic policies to determine whether they, among others, stimulate growth and private sector investments that create broad-based economic opportunities.

The other example presented at the conference came from Finland, which has had a long-term commitment to both sustainable development and to evaluation. The SDGs provided the platform for developing a vision for sustainable development and Agenda 2030 that encompasses all sectors, public and private, and civil society. It also involves a commitment to accountability towards citizens to “providing a systematic, open, transparent, inclusive and participatory follow-up to and review of the implementation of Agenda 2030”.  Most interesting from a point of view of policy coherence, this approach promises to provide cohesive feedback on the collective development footprint of Finland. It also creates platforms that link policy-makers from line ministries in Finland with counterparts in partner countries rather than only the development agency. 

The task of building such an evaluation system is tough, not just from a technical point of view. More so, it challenges the traditional world order and norms, and when functioning well raises questions that often remain untouched because of institutional silos. As a long-time evaluator, I hope that countries – policy-makers, institutions, civil society – will have the courage to step up to the challenge of evaluating policy coherence as a first step in making better informed choices to achieve the SDGs.

Comments

Submitted by Jerri Husch on Sat, 01/07/2017 - 09:09

Permalink

Excellent commentary and much needed. These kinds of "alignment' issues need to be continually brought back to the larger issues inherent in "social change" work. Too often the focus stays at the micro level with lengthly debates about indicators that lose the bigger picture.

Thanks!

Add new comment