Independent versus self-evaluation - is there a place for both?
Why spend money on self- and independent evaluation? Couldn't we improve the value-for-money equation by saving on one or both of them?
Why spend money on self- and independent evaluation? Couldn't we improve the value-for-money equation by saving on one or both of them?
By: Caroline Heider
As we come back to our blog series on the value-for-money of evaluation, let's start with the question why we have to spend all this money on self- and independent evaluation. Couldn't we improve the value-for-money equation by saving on one or both of them?
Monitoring and evaluation spending affects the cost of business. Business efficiency has been one of the reasons why institutions and private sector entities have been hesitant to invest in monitoring, and even more so in evaluation. To them, the cost associated with evaluation does not pay back in benefits or profits derived from the information. And that's likely even less so when there appears to be a double cost of separate self- and independent evaluation.
In an ideal world, we would have one evaluation system that is completely streamlined and efficient, produces the right kind of information at the right time, and does not shy away from difficult or unwanted messages. But, as many of us development professionals know, that is a tough call.
Value of Self-Evaluation. Self-evaluation has the advantage of being much closer to the action. It therefore has a great potential to promote learning, if institutional incentives are right. The value of self-evaluation is two-fold: one, fixing problems as they arise, and two, making sure avoidable mistakes are not repeated. In both cases, unnecessary expenses can be spared, harm avoided, and relationships maintained or even improved.
Value of Independent Evaluation. This second tier of evaluation derives its value exactly from being arm's length. It gives credibility to the institution as a whole which increases trust and confidence of shareholders and others. In addition, it can make up (in part) for weaknesses in the self-evaluation system. Importantly, independent evaluation produces value by, one, helping to fix systemic issues, and two, evaluating a set of interventions from a different perspective to generate new insights. This can lead to greater institutional efficiencies, and to identifying opportunities that would not be seen from the day-to-day operational perspective.
Estimating Value Added. The value of these gains from evaluation (self and independent) are hardly ever estimated. On the operational side, more effort is spent on pressing for higher quality evaluations or defending results and performance of past programs. As evaluators we are still struggling to see whether our evaluations are effective, meaning recommendations are taken up and implemented. This leaves limited space to determine the value added from evaluation.
Cost of Evaluation Systems. A common lament from operations managers is that the cost of collecting data outstrips monies available for managing the project. This is even more so in humanitarian assistance or the private sector. The trade-off seems to be that instead of evaluation (direct and indirect costs), money could be spent on serving more people better, making a project work more efficiently, or reducing the cost of a private sector deal. To some extent, I am sympathetic to this position. It is not always obvious how evaluation systems (self or independent) speak to the information needs of decision-makers or program implementers. Data collection requirements seem endless, reports are unwieldy, and messages nuanced. Simple, sharp and clear answers about "what's next" don't emerge easily. And, often there are calls for more and more data collection.
Yes, there is space, even need for both self- and independent evaluation systems. They complement each other. Better self-evaluation will result in greater efficiencies of independent evaluation.
But, there is also need for action from all sides. Decision-makers and implementers need to work with the designers of self- and independent evaluation systems to clarify information needs. Challenges will arise as different needs have to be weighed against each other. For instance, immediate business needs might not pay due attention to development outcomes. Nonetheless, there is probably room for finding a better balance to achieve:
Greater efficiency in self- and independent evaluation systems whereby data collection is "right sized",
Greater effectiveness of self- and independent evaluation systems whereby evaluation evidence is used in decision-making, corrective action, and learning for future policies and programs,
Filling information gaps by developing a deeper understanding the value added of evaluation evidence, the improvements it has (or could have) led to, and resultant implications in terms of costs and savings.
Comments
Add new comment