In some instances, commissioning one standalone evaluation is all that's needed. But increasingly organizations across the spectrum are following the long-standing practices of the multi-lateral development banks: they institutionalize evaluation functions. As institutions embark on establishing evaluation functions, they need to ask themselves what does success look like: what difference does the evaluation function make and how can they get the most value for the money spent on institutionalizing an evaluation function?

My starting point is that evaluation produces evidence and knowledge about what works and leads to better informed decisions and actions and thus generates better results. In a previous blog in this series on the value for money of evaluation, I discussed the theory of change of an evaluation.

Simple Theory of Change. When institutionalizing evaluation, the value generation has to add up beyond individual evaluations. As illustrated in the chart below, a simple theory of change involves five steps. It starts with the value addition that can be generated when evaluations complement each other, create a deeper understanding, and lead to institutional learning and accountability. Or, an institution'€™s credibility may receive a boost when it can demonstrate to stakeholders such as donors, civil society, and partners that it has a strong evaluation function and learns from its insights. Shareholders benefit from independent evaluation in governing the institution with an eye to increase development effectiveness, which benefits client countries and ultimately their citizens.

Let's explore each of the steps.

A Portfolio Approach to Evaluation. The choice of what gets evaluated is particularly important for to increase the value-for-money of evaluation. When evaluations are strategically chosen to influence the institution'€™s performance and outcomes, the institution can gain higher returns. Making those strategic choices is not always easy. One has to anticipate where evidence is most needed and can make the biggest difference a couple of years down the line. In addition, the evaluation function needs to balance two key considerations.

  • Coverage of the institution's work to generate evidence and evaluation of the health of the institution as a whole; 
  • Strategic issues, which once deeper evidence is generated, can be the biggest change agents for institutional results and performance.

This seemingly impossible combination can be achieved by the right combination of types of evaluations, which focus on different units of accounts (from projects through country strategies to policies and corporate issues) with different types of evaluations (validation of self-evaluation, independent evaluation, and synthesis products that build on existing evaluation evidence). The choice and especially combination of different types of evaluations can be a major driver in value and cost, with choices increasing or reducing both value and/or money.

Institutional Learning and Accountability. A key driver of the value of evaluation rests in the hands of the institutions and their top management. The shared interest of evaluation functions and their host institutions should center on increasing the value of evaluation, with evaluation functions playing their role in generating strategic, high quality, and timely evaluations, and top management ensuring that institutions are both open to learning and change, and to being accountable for results and performance. To achieve this goal, institutions increasingly designate a focal point to interact with evaluation. These arrangements can help optimize the value of evaluation -€“ for instance, by institutionalizing learning, taking corrective action, developing new policies and programs that build on lessons from the past -€“ while also incurring costs.

Institutional Credibility. Strong independent evaluation serves the credibility of the institution overall. Stakeholders, shareholders, clients, donors, partners, civil society -€“ recognize an institution'€™s willingness and ability to be critiqued, learn from experience and take timely corrective action, and be accountable and transparent. The value of evaluation in this context is that it is independent and produces high-quality work, which can increase the confidence external stakeholders have in the institution. In addition, clients of the institution can see independent evaluation as a source of learning for themselves as well as use evaluations to hold the institution accountable.

Informed Governance. The value of independent evaluation is particularly important for governance. Independent evaluation informs the governing bodies -€“ shareholders or member states -€“ about results and performance of the institution as a whole and in specific areas. Evaluation informs the discussion of governing bodies with management about future directions, reform agendas, comparative advantage, portfolio performance, and the like. Evaluation as an institution provides a continuous input to these debates through its ongoing work and with a system to follow-up on past evaluations. Such a system involves tracking follow-up actions to recommendations, but also reviewing new or revised policies and strategies to assess whether lessons from the past have been embedded in them.

Greater Development Effectiveness. The ultimate goal of evaluation is to enhance the development effectiveness of the institution. Just like with an individual evaluation, the transmission lines between evaluation and changes in development effectiveness of an institution are long with many intervening factors. Optimizing value-for-money of evaluation might require meeting all of the above points and still managing risks that arise from changing context and other institutional factors  that might run counter to realizing the full value of evaluation.  

Comments

Submitted by Dorgelesse VEGLANO on Wed, 03/30/2016 - 02:25

Permalink
I have truly no comment to make, I want to just appreciate . Thank you .

Submitted by Caroline Heider on Fri, 04/08/2016 - 00:03

In reply to by Dorgelesse VEGLANO

Permalink
That's nice. Thank you!

Submitted by Glenys Jones (… on Wed, 03/30/2016 - 00:56

Permalink
Well put. Institutionalising performance monitoring, evaluation and reporting is an important step in improving (and/or optimising) the achievement of objectives and desired long-term outcomes. Tasmania (Australia) has made significant progress in developing and institutionalising an evidence-based adaptive management approach to its national parks and reserves. For more information and examples of evaluation reports, go to www.parks.tas.gov.au/monitoring.

Submitted by Caroline Heider on Fri, 04/08/2016 - 00:04

In reply to by Glenys Jones (…

Permalink
Excellent news! Thanks for sharing.

Submitted by Agaje Mekonnen… on Wed, 03/30/2016 - 22:32

Permalink
I believe that indicators are key parameters to evaluate value for money development program effectiveness performance.

Submitted by Anthony Nkwocha on Thu, 03/31/2016 - 00:05

Permalink
I completely agree with you. Nice piece of well written thoughts. The role evaluation plays in determining relevance and appraising projects' realities is undeniable. However, we must watch that, we don't in any way ploiticize the institutionalization process by focusing on what sells us better, rather than reality of a social good. Very educative and interesting post. Thank you, Ma.

Submitted by Caroline Heider on Fri, 04/08/2016 - 00:06

In reply to by Anthony Nkwocha

Permalink
Thank you for the nice feedback, and for the point of keeping the eye on evidence-based decision-making for the greater social good. An important reason for doing evaluation work!

Submitted by Leticia Artiles on Fri, 04/01/2016 - 05:36

Permalink
It´s an excellent and useful article

Submitted by Caroline Heider on Fri, 04/08/2016 - 00:06

In reply to by Leticia Artiles

Permalink
Thank You!

Submitted by Atul Mohan on Sat, 04/02/2016 - 04:39

Permalink
Appreciate your views and approach. Honest independent evaluation tells exactly where does the project stand and in which direction it needs to change its course and move on to reach its destination.

Submitted by Caroline Heider on Fri, 04/08/2016 - 00:07

In reply to by Atul Mohan

Permalink
Exactly!

Submitted by Rosemary Vega on Tue, 04/05/2016 - 00:50

Permalink
I like your ideas, I will keep them. When reading you, I thought on evaluation as an expression of the institutional culture and independent evaluation as a big piece.

Submitted by Caroline Heider on Fri, 04/08/2016 - 00:08

In reply to by Rosemary Vega

Permalink
Thank you Rosemary. Appreciated!

Submitted by James Theuri on Mon, 04/11/2016 - 22:12

Permalink
This is a very good article. My comment is that for projects to achieve intended objectives, evaluations will need to be demystified. Many a team member see evaluation as a foreign activity and imagine that it should be carried out by people outside the organisation. If evaluations are well institutionalised, reporting performance every so often, e.g. on quarterly basis or even on monthly basis, people get used to preparing evaluation reports. This brings about the intended change in mindset and improves performance.

Submitted by Caroline Heider on Thu, 04/14/2016 - 04:20

In reply to by James Theuri

Permalink
Good point, James, but our evaluations show that even when there are regular performance reports, they do not lead to the kind of ownership you talk about.

Add new comment