In times of resource constraints – have there ever been days without? – one would think “efficiency” is at the top of the agenda for almost everyone. Unfortunately, we have seen limitations to this evaluation criterion in definition and above all in practice.
By: Caroline Heider
TWEET THIS
Evaluation could complement the estimates made at design with data on actual costs and benefits.
The hidden costs of social and environmental impacts need to factor into the cost of interventions.
Evaluation methods for efficiency will need to become more sophisticated to deal with waste.
One could think that evaluating efficiency does not matter, in spite of resource scarcity and the ever increasing need for improved cost-effectiveness. However, if anything we need to get better at assessing efficiency for a number of reasons.
Efficiency is often defined in terms of “measuring the outputs – qualitative and quantitative – in relation to the inputs. It is an economic term which signifies that the aid uses the least costly resources possible in order to achieve the desired results. This generally requires comparing alternative approaches to achieving the same outputs, to see whether the most efficient process has been adopted.” (OECD/DAC key terms for evaluation)
Way back when I was evaluating development projects at the Asian Development Bank, we used a definition that focused on the economic efficiency of projects; a practice shared across multilateral development banks. It is implicit in the above definition (note the reference to the economic term and least cost models). It is calculated as economic rate of return, and uses a “net present value” of the investment – a standardized rate – to determine efficiency against alternative investment opportunities. This approach goes beyond the narrow definition of efficiency that compares input-output relationships, maybe more often used in grant-funded aid projects.
But, as pointed out in an IEG evaluation of 2010, the practice of Cost-Benefit Analysis has been on the decline at the World Bank for several decades, dropping from 70% of projects including calculations of economic rates of return in the 1970s to 25% in the 1990s. This drop was in part explained by an increasing number of projects in sectors for which this kind of cost-benefit analysis was not feasible. Even when undertaken, the results of the analyses were not used in deciding whether to fund a project or not, undermining the rationale for undertaking the calculations in the first place. Another study, commissioned by the German Ministry of Development Cooperation, compared methods to assess efficiency used both at appraisal and evaluation. It concluded that many methods were little known and used.
One could think that evaluating efficiency does not matter, in spite of resource scarcity and the ever increasing need for improved cost-effectiveness. However, if anything we need to get better at assessing efficiency for a number of reasons.
The systems approach that complexity requires us to use, has the potential for comparing different intervention options and a combination of them. Let’s assume we could model a development challenge just like the US Army had the conflict in Afghanistan (see TED Talk by Eric Berlow), it could allow development practitioners to identify not only options that would generate the highest impact, but also options that are more or less costly and determine the most cost-effective package of interventions. Evaluation could assess the quality of those assessments, and whether they were used in decision-making, as well as complement the estimates made at design with data on actual costs and benefits at the time of evaluation.
Less futuristic, there is a great need to factor into the cost of interventions the hidden costs of social and environmental impacts. Today, the cost of pollution is more often factored into investments, especially when mitigating measures have to be taken or technology has to be adapted to clean up pollutants rather than releasing them unfiltered into the atmosphere. But more will need to be done in evaluating the efficiency of these investments over alternative choices.
Finally, evaluation methods for efficiency will need to become more sophisticated to deal with waste. Losses, such as in electricity or water distribution systems do get accounted for in the evaluation of economic efficiency. However, as the SDGs call for a change in consumption patterns, methods will need to develop a better understanding of the consumption patterns implicitly (and hopefully increasingly explicitly) that an intervention promotes, determine when they are wasteful, to signal the need for rethinking of incentives.
Is evaluation ready to rise to these challenges? Comment below and share your opinion with us.
Have we had enough of R/E/E/I/S?, Is Relevance Still Relevant?, and Agility and Responsiveness are Key to Success
