What is the Value of Independent Evaluation?
Organizations like drivers can suffer from blind spots. Independent evaluation can help provide a broader view.
We all evaluate all the time. When you cross the street you evaluate the speed of the oncoming traffic and maybe even the degree of madness of the drivers. As a driver you scan pedestrians to assess which of them may dash across the road at an untoward moment.
In either situation if you make the wrong choice an accident happens. These scenarios elicit what Daniel Kahneman in his best-selling book Thinking, Slow and Fast calls a “system 1 response” where our pre-programmed knowledge leads us automatically to our choice.
The objective for the driver and pedestrian is simple: avoid an accident. Achieving complex objectives is much harder - for many reasons:
Complex relationships: Complex development processes involve many people. Common, agreed goals are often open to interpretation. Even people on the same team might differ widely in their views and some will work together better than others. Self-evaluation might uncover those diverse views or, if done from the perspective of one stakeholder, miss them.
Complex issues: The more complex the issues the more difficult it is to evaluate what is working, what isn’t and to find the reasons why. Complex projects, policies and strategies involve many factors that lead to or prevent success. Those who manage and self-evaluate the program will know many of these factors as they deal with them every day, but they may also lack the perspective to see larger patterns and interrelated issues.
Reality check: The lofty goals of our projects and policies are often ideals that we would love to achieve, yet circumstances may limit our ability to succeed. Sometimes pressures – real and imagined – may force institutions to promise more than what is doable. Self-evaluation might take a sympathetic view of the level of effort but not see opportunities to close the gap between the goals and the actions taken to achieve them, or it may be constrained in dealing with those pressures for setting overly ambitious targets.
Blind spots: We all have blind spots. If we work on something for a long time with conviction it can become difficult to see warning signs that might be spotted by a more impartial observer. What is true of individuals is also true of institutions. Culture and norms set a tone that determine which issues can be discussed, which problems can be resolved, and which ones can’t. People know the problems and which ones are “off limits”. Self-evaluation can be constrained by self-censorship in seeking to remain within the institutional boundaries.
Accountability & Learning: Knowing these limitations, stakeholders – donors, civil society and those affected by projects, policies and strategies – want an independent view of what works and what does not to hold institutions accountable. Only by facing those limitations – from inadvertent blind spots to norms that impede raising critical issues – can learning take place and limitations be overcome.
So, what is the value of independent evaluation? Are our “system 1 response” mechanisms sufficient to check the blind spots or do we need a “system 2” – a slower thinking mechanism like independent evaluation in order to: