Our goal is to stimulate a richer, more strategic discussion with World Bank Group senior management that gets to the bottom of the underlying systemic issues that individual evaluations cannot fix.
As evaluators, our ultimate goal is to help our clients improve the development effectiveness of their work by providing useful insights, based on our assessments of project results and performance. Making sure that recommendations are implemented is part of that job. And, in IEG’s annual Results and Performance report of the World Bank Group’s operations we review the extent to which the Bank Group uses our recommendations to take corrective actions. To do this, we have a developed a system: the Management Action Record.
How does it work?
Once a year, for four years, IEG confers with colleagues of the World Bank Group on follow-up actions to recommendations in our evaluations. The process involves a review of progress, a self-assessment by the World Bank Group, and a validation by IEG. The resultant World Bank Group and IEG ratings are tracked over years to assess how much progress has been made.
What does the latest review tell us?
In 2016, IEG tracked agreed actions for 150 recommendations drawn from 26 evaluations produced between FY2012 and FY2015. While there has been progress across all the tracked recommendations, there remains a significant gap between IEG’s progress ratings and the Bank Group’s own assessments of how much progress has been made. In some cases, this reflects recommendations where Bank Group Management only partially agreed with IEG’s recommendations or where they argued that IEG’s recommendations are not feasible, or that they lack the budgets and time resources to deliver.
What happens after four years?
Of the recommendations made since 2008 that are four years old or older, IEG rated 77 percent substantial, high, or completed. The figure below shows the trend line for evaluations tracked over the last four years to illustrate the trajectory.
After more than a decade of using the Management Action Record as a tracking system, we have also learned that it takes more than tracking to make a difference.
First, actions do not automatically resolve the issues. The system helps us track whether the Bank Group has taken the promised actions. But, that has not necessarily led to a resolution of the underlying issues found in our evaluations. For instance, we continue to identify issues with results frameworks that respond to real client needs and monitoring and evaluation systems that help with timely course-correction – something desirable for the Agile Bank and the Solutions Bank Group – or internal knowledge transfer to improve understanding of country context or internal work quality. If done right, working with our clients to define right response actions can help ensure that underlying issues are addressed.
Secondly, a rich and open discussion with key stakeholders to understand evaluation findings and agree on actions can lead to greater effectiveness of evaluations. Such an engagement process often leads to more buy-in and better responses in terms of what corrective actions to take. In the case of the World Bank Group, we are piloting different ways to draft and consult on recommendations and action plans; something we will report on in a future blog.
Third, we will be launching three pilots to try a strategic dialogue to discuss recommendations of several evaluations. We expect several benefits from this process. By taking this big picture approach, our goal is to stimulate a richer, more strategic discussion with World Bank Group senior management that gets to the bottom of the underlying systemic issues that individual evaluations cannot fix. In addition, it will help us in IEG understand our collective recommendations and whether we need to make them more influential. The three upcoming pilots involve different sets of stakeholders: one around the World Bank Group’s work on private sector development to be discussed with operational VPs in the World Bank Group, a second one that is looking at strengthening monitoring, evaluation and learning systems across the Bank Group to be discussed with the Board and Managements, and a third that will explore the Bank Group’s progress on environmental sustainability, which will involve responsible leadership.
By championing responsive action plans, increasing engagement, and initiating strategic dialogue on issues raised across multiple evaluations, IEG expects to contribute even more to increasing World Bank Group development effectiveness.