When Atul Gawande, the well known surgeon and writer, spoke at the World Bank in October, one of his many excellent examples struck me in particular. He contrasted the use of anesthesia with that of hand-washing. Both ultimately supporting the goal of the surgeon – to save the life of the patient – but one more immediate and attractive, and thus easily accepted. No, it wasn’t the hand-washing but rather anesthesia that won the contest. Why? Atul attributed it to the immediate gratification – patients do not feel pain or scream under anesthesia, which reduces the stress on the surgeon – as compared to the gap in time and many other intervening factors that play a role in post-surgical infections and subsequent deaths. A line of sight too long even in a face-to-face relationship between patient and surgeon?
So where does this leave us, the development practitioners who want to eradicate poverty and boost shared prosperity? The distance – physical, time, cultural, income disparities – between the poor we aim to help and the realities in World Bank offices are much larger than those between a patient and a surgeon. Many factors intervene – from people, power and politics, institutions, assumptions and models about growth, development and distribution, to natural resources, calamities, and others – some in our favor, others stopping us dead in our tracks and annulling all of our efforts. It seems more like Edward Lorenze’s butterfly effect: small changes in one place can have large effects in others, some of them deliberate, others unintended.
With so many actors in play (from colleagues and supervisors to shareholders, counterparts, partners, and people affected by projects), and factors most of which are beyond their control, what is the development practitioner empowered to do, what is he accountable for, what can she influence?
World Bank President Jim Kim is committed to creating metrics and data systems that will help everyone grasp what difference they make to achieving the two new goals and help him and his senior team know whether everything is on track to meet the interim target of halving poverty by 2020 to 9%. This means shortening the “line of sight” to illustrate the effects of changes in behavior on the World Bank Group’s development impact. It speaks to the new Strategy’s commitment to align all activities and partnerships towards the two goals.
All of this speaks to the heart of us evaluators. We love clear goals, accountability, line-of-sight, and metrics. The stronger the self-evaluation system is, the better we can validate its findings and the more we can focus our energy on areas where independent evaluation adds greater value and has a comparative advantage.
But: these changes are not without challenges for us evaluators. For instance, aligning all activities and resources singularly with the two goals requires methods that assess whether at the outset, during implementation, and the end this alignment was realized and whether it generated better results.
Our main indicator – relevance – is in principle appropriate to assess alignment. But, the two goals are such that multitudes of pathways are possible, leaving the door open to a rather broad interpretation of alignment that could result in a generic assessment of relevance.
The second cue, that the Bank Group is committed to work with client countries to address their toughest challenges should help narrow the window, but will challenge us to go beyond evaluating whether the objectives were achieved to assessing whether the right choices were made, whether they were adjusted over time to keep focused on the toughest choices, and how success is measured: is “having tried and failed” better than “having never tried”?