In 2009, the President of IDEAS invited me to give the keynote address about evaluation capacity development at the Global Assembly. The address was subsequently published in Influencing Change. The framework I presented found its way into the thinking of others. It led to a paper jointly written with respected colleagues like Marco Segone and Riitta Oskanen Towards a shared framework for National Evaluation Capacity Development. The co-authors brought together evaluation networks of bilateral, multilaterals (both the United Nations and the Multilateral Development Banks), and voluntary organizations. The paper included, among others, a call for creating an enabling environment for evaluation, an idea taken up by EvalPartners and others for their effective work with parliamentarians and beyond. 

Six years on, this year saw an unprecedented number of evaluation conferences designed to position evaluation  to help shape the agenda for the future. The advocacy campaign for evaluation was deliberately planned to coincide with the end of the Millennium Development Goals (MDGs), the adoption of the Sustainable Development Goals (SDGs) and two global conferences - in Addis Ababa and in Paris - organized to discuss how the new agenda will be financed.

And it is the Financing for Development conference in Addis that heightened opportunities for evaluation capacity in partner and client countries. The premise is clear: domestic public finance is on the rise in absolute terms and as a proportion of investments in development. It comes with obligations for good governance. Actors in public and non-governmental domains will want to learn whether investments are producing expected results and contributing towards achieving the SDGs. And they will want to draw lessons for replicating success while avoiding unnecessary repetition of mistakes. And the SDGs, framed as a global commitment but with local action, have further implications for evaluation at large and for evaluation capacities in particular. 

My prediction is that by 2030, countries will be evaluating not only programs funded by domestic public finance, but those financed by external partners, possibly including the private sector. 

Was EvalYear sufficient to unleash what's necessary for such a vision to come to fruition? It set out an aggressive agenda. It aimed to engage policy makers in governments and parliaments to understand the importance of evaluation to create an enabling environment for evaluation. This is an important building block to promote and develop evaluation capacities.

But, as I explained in my contribution to Influencing Change, processes are just as important for capacity development as all else. Here are a couple of ingredients that have proven effective - through action research and validated through evaluation - for capacity development:

  • Drive from Within: ownership, leadership, and collective action. Leading countries such as Mexico and South Africa have invested in evaluation capacity for some time. They are being joined by others, who are now recognizing the importance of evaluation in increasing their development effectiveness. The advocacy work of this year has enhanced awareness of the usefulness of evaluation as a tool for decision-makers rather than a donor requirement. This is important to find champions, who can promote evaluation within countries and network with likeminded policy-makers. 
     
  • Participatory Diagnostics: understanding capacity gaps and needs together. Participatory diagnostics are a deliberate process of understanding what capacity is desired and needed, where the weaknesses lie, and how they can be overcome. The diagnostic phase becomes, if done in participatory manner, already part of the process to develop capacities. Evaluations of capacity development have, however, shown that diagnostics are done as expert opinions and result in blueprints that are not necessary understood, shared, or owned by those whose capacities are to be developed.
     
  • Order Chaos: having a plan, and the ability to deviate from it when opportunities arise. Probably the most complicated aspect of capacity development is that there is a need for clear commitments to outcomes - capacities that should exist at the end - and the flexibility to adjust constantly as opportunities or obstacles arise. For instance, a change in stakeholders might bring in a champion and free up resources that were earmarked for advocacy. Instead, these could be reprogrammed to help build institutional systems.

This level of sophistication of capacity development requires strong ownership within countries. And, I would argue that this will be the only way to integrate the many inputs from bilateral, multilateral, and non-governmental partners who are supporting evaluation capacity development in various ways.

What's next? Getting some countries already committed to evaluation to lead the way and demonstrate how it can be done, and dialogue with the next wave of countries ready to engage on this journey.

Comments

Submitted by Habib Mayar on Thu, 11/26/2015 - 02:47

Permalink
Agree with every proposition, particularly the notion of "driving from within" to be central at evaluation and communication of results. This is particularly a critical element in countries affected by war and conflict where institutional capacity on monitoring and evaluation needs to be strengthened. The visibility or results in the form of success or otherwise is a necessary element to the legitimacy of resilience efforts. In most cases results are monitored and evaluated with common well developed standards unfortunately. Those standards are too rigid to capture the tiny, but substantial changes which could help in changing perception. Not only that, but donors are fragmented even at country level. The evaluation and monitoring piece of work is limited to their own projects and piece of work happens exclusive of the government entity hosting that project and program. For the "global commitment, but local action" to be realized, the external actors (IFIs, multilateral and bilateral) have to harmonize and unify their resources both at country level and regional and global levels. While the cost of SDG monitoring might be huge, but there should be some investment in the state institutions to take forward the work of evaluation in the spirit of ensuring accountability.

Submitted by Caroline Heider on Wed, 12/16/2015 - 22:37

In reply to by Habib Mayar

Permalink
Excellent comment, Habib! Let's continue our conversation to see how we can make at least some of these ideas come true.

Submitted by Temple Jagha on Thu, 11/26/2015 - 22:10

Permalink
Evaluation capacity and evaluation capacity development are not new to most parts of African anymore, in their bid to achieve development results. There have been a myriad of monitoring and evaluation training activities for different sectors including the public/government, private/commercial, and non-governmental sectors. However, what I think is the major issues, is coordination. There are lots of development projects being implemented ta the same time by different entities, many of these projects are similar, and many are working towards affecting the same SDG (formerly MDG). These projects are all implemented independent of the others, and each entity attempting to claim attribution of results at population level. The competition among partners and projects is one reason that evaluation has not made the in-depth impact that it should have, in a country like Nigeria. DFID, USAID, GTZ, PACT, etc all have, for instance, MNCH projects in the northern parts of Nigeria. While project implementers meet with government and know what others are doing on the ground, there is not much leveraging. USAID and DFID clearly encourage collaboration to achieve higher results, but this does not happen fully and projects remain stand-alone. In all of this, evaluation is reduced to data collection for individual projects, and results regarding the projects outputs instead of a collaborative means of evaluating the overall impact of individual but similar projects on the population. Evaluation is conducted only to assess impact of individual projects. On the other hand, we all agree that more than one project will contribute to a single SDG. One project alone cannot achieve the level of change expected on each SDG, and a collaborative approach to evaluation, based on the contribution of all similar projects is the only way that we can get some sense of changes in the SDGs towards achieving the agreed targets. If the MDGs were too complex, and the SDGs have been simplified to some level, the same individual project approach to evaluation will hamper the way results to the SDGs are identified and reported. While I do not write anything new, evaluation capacity is relatively low because it is not of paramount importance in the achievement of SDGs. Evaluation capacity should be a SDG by itself, with resources devoted to it and some level of independence granted it so that comprehensive projects can be implemented towards improving evaluation capacity in Nigeria and Africa.

Submitted by Caroline Heider on Wed, 12/16/2015 - 22:43

In reply to by Temple Jagha

Permalink
Temple, thank you for your insightful comment. I agree with many points you made: the need for more effective coordination to avoid duplication, and raising the bar for evaluations to focus on more than just the one project. In some cases, doing sector evaluations can help, especially when different donors/stakeholders in the country agree on working together on that. Is there a strong authority in Nigeria who could advocate for a sector-wide evaluation of a particular issue, like MNCH which seems dear to you? In some countries, donors have coordination meetings -- putting to them the idea to evaluate their efforts together might be another idea to try out.

Submitted by Muhammad Alkali on Sun, 12/06/2015 - 04:49

Permalink
Can we make evaluations simple, integrated and affordable? If yes my 100% support

Submitted by Caroline Heider on Wed, 12/16/2015 - 22:47

In reply to by Muhammad Alkali

Permalink
Muhammad, it really depends a bit on the subject of the evaluation. Some are easier to do simple and integrated than others. But, point well taken that evaluation has to be affordable.

Add new comment