Who needs evaluation capacity? Years ago the answer to this question would have been the donor community.  This situation has changed dramatically in the last couple of years. Many more countries are setting up evaluation offices as part of their own systems. They – from governments to citizens – want to know how well policies and programs work, whether funded by an external donor or government revenues.

This trend was evident at the Fourth Global Forum of the Regional Centers for Learning on Evaluation and Results (CLEAR) Initiative in Mexico City, where I had the privilege to meet and share ideas with leaders from around the world about their ambition to build evaluation into their systems for managing public sector service delivery, finance, and beyond. It is a demonstration that governments recognize the importance of evidence and greater understanding of development processes. With a capacity to evaluate, countries are able to make better informed decisions – including taking calculated risks – about the management of the public sector as a whole and public finance in particular. Evaluation contributes to increased ownership of development processes, responsible governance, and timely corrective action when policies, programs, or institutions do not perform at their best.

Country  ownership and  evaluation champions are essential for the process of developing evaluation capacity. These champions can be policy-makers who see the true value of evidence in the decision-making and evaluators as the interlocutors with policy and decision-makers in their countries. They will ensure that evaluation focuses on priority needs and that it is taken into account. Champions are also essential for engaging with the many agencies that aim to develop evaluation capacities identifying the bottlenecks and the additional needs for support. They are the drivers of networks where leaders of strong evaluation functions reach out to debate each other’s challenges and solutions, as well as include peers in countries that just started their journeys in the evaluation capacity world.

Developing capacity involves a diagnostic process that is driven by those local champions. They should be able to call on other leaders in a similar situation and in the evaluation community to help them identify capacity needs and determine gaps and ways to close them, just as took place at the South-South Roundtable in Johannesburg that brought together leaders  from Argentina. Benin, Brazil, Colombia, Chile, Costa Rica, Guatemala, El Salvador, Niger, South Africa and Zambia.

Their ownership is essential for creating a shared vision, clear goals and targets, and a monitored, adaptable implementation plan. Implementation will involve typical elements of training, expert advice, and other inputs, and increasingly dialogue among leaders to develop:

  • the enabling environment, which both sets the policy or authorizing context and signals demand for evaluation,
     
  • the institutions, which require they understand their clients, develop services, methods systems, and the human, technical, and financial resources to deliver services, and
     
  • people, who need skills, knowledge, and networks to perform at their best.


In recent years the number of evaluation capacity development efforts has exploded. The good news: more attention, more resources. The challenge: generating more and better synergies between these efforts so that they collectively produce more than the parts can deliver individually. Groups like the Development Assistance Cooperation Evaluation Network of the Organisation for Economic Co-operation and Development  and  initiatives like CLEAR or EvalPartners rally diverse stakeholders around a common goal, which is a good start.

How else to support these efforts?

Fostering an enabling environment for evaluation can take place through dialogue among high-level decision makers and evaluators to increase awareness of and demand for evidence from evaluation. Building evaluation into public sector/finance management programs would give a strong impetus to an enabling environment that asks for and uses critical evaluation findings.

Developing institutions. Take project-related Monitoring and Evaluation units:  instead of developing these units as part of a project management structure, they could be used as a platform for developing institutional evaluation capacities as an integral part of the government’s public administration. These units, situated in line ministries, when networked with the national statistical offices would form part of an institutionalized system that generates monitoring and evaluation data for all government investments rather than projects, and feed information into the national statistic system. In addition,  it is important to integrate the evaluation data these institutions generate  into specific decision-making processes with a clear understanding how evaluation evidence can and should inform choices, and a strategy to ensure evaluation services provide necessary evidence. A network of institutions will ensure that the system overall is efficient and connected with national statistical services.

Building people’s skills and knowledge receives a lot of attention, albeit with short-term measures. To close the demand gap for highly qualified evaluators, a long-term solution lies in investing in tertiary education that incorporates evaluation in faculties such as public administration, both for those who will become policymakers and policy implementers and need to understand evaluation evidence and its use in policy making and implementation, as well as graduate programs for evaluators to equip them with the necessary skills to deliver high quality evaluations. Working with a group of leading universities in partner and client countries to create a network will also help in developing comparable curricula, professional standards, and eventual professionalization of  this young profession.

Comments

Submitted by Shoghik Hovhannisyan on Wed, 12/11/2013 - 06:07

Permalink
Dear Caroline, thank you very much for bringing up this topic into a discussion and providing such a comprehensive overview of steps to move forward. I also have a few thought to add to this discussion. As IEG states in its mission evaluation can enhance development effectiveness and therefore building evaluation capacity in developing countries is an essential ingredient in reducing poverty and ensuring inclusive growth. Currently resources spend on evaluations in developing countries are minuscular and they are also relatively small in multilateral development agencies as a share of their budgets. While sophisticated evaluation techniques and advanced evaluation skills are being applied in developed economies using rich information set, poor countries tremendously lag behind. In our daily work we often encounter lack of data to conduct evaluations and use proxies to make best of what we have. Building evaluation capacities and shaping evaluation culture across institutions in WB member countries will improve transparency and efficiency of public sector, create a strong statistical database, help us better understand development challenges in various country settings, and avoid facing a black box in designing development policies. With current transformation of the WBG towards evidence based organization I believe evaluation will become a priority which will have a multiplier effect on our client countries. From my experience in the Ministry of Finance and Economy of Armenia I personally witnessed how guidance from the WB in preparing the Poverty Reduction Strategy Paper and designing a proper Monitoring and Evaluation Framework helped the government improve its strategic focus, better understand challenges, and plan policy measures in a results-based manner. Although most of the time it is about resources, but not always. WBG’s diverse group of client countries requires different approaches and solutions. In countries with no financial constraints such as resource-rich countries, new European Union member counties, or BRICS the agenda is about knowledge sharing and skill building. I think the WBG and is well equipped to address this important issue in near future.

Submitted by Caroline Heider on Tue, 12/17/2013 - 23:57

In reply to by Shoghik Hovhannisyan

Permalink
Shoghik, many thanks for your comprehensive posting and for sharing your own experience in Armenia. Very useful.

Submitted by Junaid H S Jadoon on Sun, 12/15/2013 - 23:51

Permalink
Dear Caroline, it is really a fruitful discussion and your suggestions in the end of blog are very pertinent particularly the independent M&E Units and taking "evaluation" as an important knowledge area. What I have experienced during my short professional career is that the realization of importance of M&E is still very meager among the development world due the perceived limited scope of this activity which in-fact not only can guide better designing but also better implementation of projects.

Submitted by Caroline Heider on Tue, 12/17/2013 - 23:54

In reply to by Junaid H S Jadoon

Permalink
Junaid, I agree with you: M&E is essential for improving implementation, observing/realizing problems early on and solving them. My sense is that there is more and more interest in M&E, and the commitment of the World Bank Group to be results-oriented, make mid-course corrections, and aim for greater development impacts are promising to give M&E a further impetus.

Submitted by Fouzia Rahman on Mon, 11/10/2014 - 00:23

Permalink
I come across Government sponsored/dictated evaluation reports with a bottomline "ALL IS WELL" . Governments claiming to be democracies have dictatorial approach toward expected findings of evaluations and prefer to bring on board consulting companies with a track record of flexibility on Government demands. Pakistan's reports by GALLUP and PILDAT usually bring out findings which support Government's prior claims on development issues as well on Government's approval ratings. I consider it a waste of tax payers' money.

Add new comment