The Complexity of Estimating the Value and Costs of an Evaluation
How do we estimate, or even calculate the value for money (VfM) from any given evaluation?
Intangible value and indirect costs, intertwined in an interdependent relationship.
Over the last six months we had a great run with our blog series on Value-for-Money of Evaluation. Thanks to the many of you who viewed, shared, and commented on the blogs. Really an extraordinary participation.
Now, here comes the challenge! How do we estimate, or even calculate the value for money (VfM) from any given evaluation?
In previous blogs, I mapped out the points at which value and costs are incurred. I suggested a number of ways to increase value and reduce the costs of an evaluation. These suggestions are valuable in their own right, as they improve evaluation practice. But, they do not yet spell out how value or cost can be estimated.
Many of the costs can be calculated, for instance the direct cost of an evaluation, or the expense associated with outreach activities. But, there are indirect costs that often do not get taken into account. They are noticed in particular when evaluations uncover controversial issues and create the need for difficult discussions. These meetings are often experienced by stakeholders – people whose programs are evaluated as much as evaluators – as an additional cost. Other costs, often hard to calculate, are those of reputational risk – feared or actual – that can arise if there is no evaluation at all, or if an evaluation is inaccurate.
The value, however, is harder to calculate. An evaluation that is not timely is a missed opportunity to make better informed choices. But, estimating the value would have to compare the choices made with and without evaluation evidence, and assume the evaluation would have influenced decision-making.
In addition, choices along the life-cycle of evaluations can increase the VfM equation at some point but reduce it at others. The most obvious case is when cost-savings lead to evaluation findings that are not robust. But, just spending more money is not the answer—wise choices about evaluation design (objectives, scope, and questions) are.
Ultimately, the value of evaluation needs to be estimated through the changes that occur as a result of it.
If, for instance, a project normally generates a return of “100” but as a result of an evaluation introduces changes that increase its return to “120” the value of the evaluation would be “20” assuming all changes introduced in the project were motivated by the evaluation.
Typically, however, the value increase of a project is neither calculated nor attributed to an evaluation as easily as in this example for a number of reasons, including the following:
None of these points should stop us from attempting to assess the influence, impact and value-for-money or our evaluation work. Please share your examples, if you have done so.