Having been in evaluation for more than 30 years, the 'broken record' is disconcerting. The reaction we often get is that we don't see anything new; my response frequently has to be: it is the mistakes that are unnecessarily being repeated that necessitate that evaluators keep flagging them. The messages will 'go away' once learning has taken place.

TWEET THIS

• If evaluators are flagging the same mistakes repeatedly, the necessary learning has not yet taken place.
• Evaluators are responsible for making an effort to stimulate learning from evaluative evidence.
• More will be needed to create an environment in development institutions that is safe for dealing with mistakes, that promotes learning, and embraces adaptive management and change.

Some of our regular readers will be familiar with the OECD’s Development Assistance Committee (DAC) – a group that brings together some of the largest providers of international development assistance. For several years, IEG has been part of an evaluation network under DAC that meets regularly to exchange information, experiences and foster cooperation among the DAC member organizations.

At the last DAC EvalNet meeting, we discussed a synthesis of evaluations in refugee contexts, which the secretariat had prepared.

I was thrilled for two reasons: first, the synthesis of evaluations helps us identify common lessons and patterns that point to deeper, systemic issues that need resolving; and, second, in this case the synthesis findings had been presented to a policy-making body to inform their deliberations on setting out new policy guidance for responses to refugee crises. This guidance will now be rolled out, vetted, and adapted at operational level. What a great example for ensuring evaluation evidence is deliberately integrated into forward-looking policies and hopefully influence future decisions and practices.

However, what was frustrating was how many of the findings, lessons, and recommendations were repeating earlier ones. Having been in evaluation for more than 30 years, the "broken record" is disconcerting. The reaction we often get is that we don't see anything new; my response frequently has to be: it is the mistakes that are unnecessarily being repeated that necessitate that evaluators keep flagging them. The messages will "go away" once learning has taken place.

But, that does not absolve evaluators from making an effort to stimulate learning. At IEG, we have a whole range of activities to try to close the learning loop:

• Similar to this example, IEG's evaluation of the WB procurement policy was conducted to and influenced the new policy;
   
• the internal review processes of the World Bank Group, and the subsequent discussion at the Board or its committees, ensure that we draw on our evaluation evidence to inform the formulation of policies and strategies;
   
• we are experimenting with deep dives into our recommendations and Management Action Records for our major evaluations to understand what we have recommended collectively from a number of evaluations, what has been done, and where the frontiers are for greater learning and change;
   
• for country strategies, IEG has contributed to the WBG training courses with modules on designing results frameworks and on selectivity. In addition, we are piloting information packages that are provided to new country managers/directors to help them get up to speed on past experience in their country of assignment;
   
• for projects, where we see great opportunities to improve quality at entry/design, IEG also teaches at mandatory training courses of the World Bank and IFC, and provides training for the preparation of completion reports at exit. In addition, a series of learning engagements focuses on enhancing the understanding of outcomes and associated indicators. Together with the Knowledge Directors of the WB and of the IFC, we have incorporated relevant IEG work into information packages task team leaders receive when they start a new project;
   
• IEG's Knowledge, Learning, and Communications Department is constantly exploring ways to make evaluation knowledge accessible with a revamped website, social media, and packaging information in easily accessible formats.

Will all of this be sufficient that we will eventually learn and change behaviors?

As shown in our evaluations of Learning in World Bank Operations and of the World Bank Group's self evaluation systems, more will be needed to create an environment that is safe for dealing with mistakes, that promotes learning, and embraces adaptive management and change.

Blog image inspired by The adaptive management cycle for the Tasmanian Wilderness World Heritage Area by Alan Diduck Patricia Fitzpatrick and James P. Robson