Institutionalizing Evaluation: What is the Theory of Change?
To optimize value-for-money, it is essential to understand how evaluation influences change
In some instances, commissioning one standalone evaluation is all that's needed. But increasingly organizations across the spectrum are following the long-standing practices of the multi-lateral development banks: they institutionalize evaluation functions. As institutions embark on establishing evaluation functions, they need to ask themselves what does success look like: what difference does the evaluation function make and how can they get the most value for the money spent on institutionalizing an evaluation function?
My starting point is that evaluation produces evidence and knowledge about what works and leads to better informed decisions and actions and thus generates better results. In a previous blog in this series on the value for money of evaluation, I discussed the theory of change of an evaluation.
Simple Theory of Change. When institutionalizing evaluation, the value generation has to add up beyond individual evaluations. As illustrated in the chart below, a simple theory of change involves five steps. It starts with the value addition that can be generated when evaluations complement each other, create a deeper understanding, and lead to institutional learning and accountability. Or, an institution's credibility may receive a boost when it can demonstrate to stakeholders such as donors, civil society, and partners that it has a strong evaluation function and learns from its insights. Shareholders benefit from independent evaluation in governing the institution with an eye to increase development effectiveness, which benefits client countries and ultimately their citizens.
Let's explore each of the steps.
A Portfolio Approach to Evaluation. The choice of what gets evaluated is particularly important for to increase the value-for-money of evaluation. When evaluations are strategically chosen to influence the institution's performance and outcomes, the institution can gain higher returns. Making those strategic choices is not always easy. One has to anticipate where evidence is most needed and can make the biggest difference a couple of years down the line. In addition, the evaluation function needs to balance two key considerations.
This seemingly impossible combination can be achieved by the right combination of types of evaluations, which focus on different units of accounts (from projects through country strategies to policies and corporate issues) with different types of evaluations (validation of self-evaluation, independent evaluation, and synthesis products that build on existing evaluation evidence). The choice and especially combination of different types of evaluations can be a major driver in value and cost, with choices increasing or reducing both value and/or money.
Institutional Learning and Accountability. A key driver of the value of evaluation rests in the hands of the institutions and their top management. The shared interest of evaluation functions and their host institutions should center on increasing the value of evaluation, with evaluation functions playing their role in generating strategic, high quality, and timely evaluations, and top management ensuring that institutions are both open to learning and change, and to being accountable for results and performance. To achieve this goal, institutions increasingly designate a focal point to interact with evaluation. These arrangements can help optimize the value of evaluation - for instance, by institutionalizing learning, taking corrective action, developing new policies and programs that build on lessons from the past - while also incurring costs.
Institutional Credibility. Strong independent evaluation serves the credibility of the institution overall. Stakeholders, shareholders, clients, donors, partners, civil society - recognize an institution's willingness and ability to be critiqued, learn from experience and take timely corrective action, and be accountable and transparent. The value of evaluation in this context is that it is independent and produces high-quality work, which can increase the confidence external stakeholders have in the institution. In addition, clients of the institution can see independent evaluation as a source of learning for themselves as well as use evaluations to hold the institution accountable.
Informed Governance. The value of independent evaluation is particularly important for governance. Independent evaluation informs the governing bodies - shareholders or member states - about results and performance of the institution as a whole and in specific areas. Evaluation informs the discussion of governing bodies with management about future directions, reform agendas, comparative advantage, portfolio performance, and the like. Evaluation as an institution provides a continuous input to these debates through its ongoing work and with a system to follow-up on past evaluations. Such a system involves tracking follow-up actions to recommendations, but also reviewing new or revised policies and strategies to assess whether lessons from the past have been embedded in them.
Greater Development Effectiveness. The ultimate goal of evaluation is to enhance the development effectiveness of the institution. Just like with an individual evaluation, the transmission lines between evaluation and changes in development effectiveness of an institution are long with many intervening factors. Optimizing value-for-money of evaluation might require meeting all of the above points and still managing risks that arise from changing context and other institutional factors that might run counter to realizing the full value of evaluation.