TWEET THIS

Evaluators and clients must define together the right response actions to fix underlying issues.
Evaluators must help stakeholders understand evaluation findings through rich, open discussion.
Dialogue with management is needed when there are underlying systemic issues that individual evaluations cannot fix.

As evaluators, our ultimate goal is to help our clients improve the development effectiveness of their work by providing useful insights, based on our assessments of project results and performance. Making sure that recommendations are implemented is part of that job. And, in IEG’s annual Results and Performance report of the World Bank Group’s operations we review the extent to which the Bank Group uses our recommendations to take corrective actions. To do this, we have a developed a system: the Management Action Record.

How does it work?

Once a year, for four years, IEG confers with colleagues of the World Bank Group on follow-up actions to recommendations in our evaluations. The process involves a review of progress, a self-assessment by the World Bank Group, and a validation by IEG. The resultant World Bank Group and IEG ratings are tracked over years to assess how much progress has been made.

What does the latest review tell us?

In 2016, IEG tracked agreed actions for 150 recommendations drawn from 26 evaluations produced between FY2012 and FY2015.  While there has been progress across all the tracked recommendations, there remains a significant gap between IEG’s progress ratings and the Bank Group’s own assessments of how much progress has been made. In some cases, this reflects recommendations where Bank Group Management only partially agreed with IEG’s recommendations or where they argued that IEG’s recommendations are not feasible, or that they lack the budgets and time resources to deliver.

What happens after four years?

Of the recommendations made since 2008 that are four years old or older, IEG rated 77 percent substantial, high, or completed. The figure below shows the trend line for evaluations tracked over the last four years to illustrate the trajectory.

MAR_RAP2016.png

Explore the Management Action Record system

Explore the data behind this year’s review of the World Bank Group’s performance

Our goal is to stimulate a richer, more strategic discussion with World Bank Group senior management that gets to the bottom of the underlying systemic issues that individual evaluations cannot fix.

What’s next?

After more than a decade of using the Management Action Record as a tracking system, we have also learned that it takes more than tracking to make a difference.

First, actions do not automatically resolve the issues. The system helps us track whether the Bank Group has taken the promised actions. But, that has not necessarily led to a resolution of the underlying issues found in our evaluations. For instance, we continue to identify issues with results frameworks that respond to real client needs and monitoring and evaluation systems that help with timely course-correction – something desirable for the Agile Bank and the Solutions Bank Group – or internal knowledge transfer to improve understanding of country context or internal work quality. If done right, working with our clients to define right response actions can help ensure that underlying issues are addressed.

Secondly, a rich and open discussion with key stakeholders to understand evaluation findings and agree on actions can lead to greater effectiveness of evaluations.  Such an engagement process often leads to more buy-in and better responses in terms of what corrective actions to take. In the case of the World Bank Group, we are piloting different ways to draft and consult on recommendations and action plans; something we will report on in a future blog.

 

Third, we will be launching three pilots to try a strategic dialogue to discuss recommendations of several evaluations. We expect several benefits from this process. By taking this big picture approach, our goal is to stimulate a richer, more strategic discussion with World Bank Group senior management that gets to the bottom of the underlying systemic issues that individual evaluations cannot fix. In addition, it will help us in IEG understand our collective recommendations and whether we need to make them more influential. The three upcoming pilots involve different sets of stakeholders: one around the World Bank Group’s work on private sector development to be discussed with operational VPs in the World Bank Group, a second one that is looking at strengthening monitoring, evaluation and learning systems across the Bank Group to be discussed with the Board and Managements, and a third that will explore the Bank Group’s progress on environmental sustainability, which will involve responsible leadership.

By championing responsive action plans, increasing engagement, and initiating strategic dialogue on issues raised across multiple evaluations, IEG expects to contribute even more to increasing World Bank Group development effectiveness.

Read Chapter 3 of the Results and Performance of the World Bank Group 2016: Management Action Record

Explore the Management Action Record system

Explore the data behind this year’s review of the World Bank Group’s performance

Comments

Permalink

Dear Caroline and Brenda, I think this is absolutely the way to go. At OIOS, I am spearheading - not without some internal resistance - a working group that looks at the intersection of evaluation and change management, in collaboration with the United Nations Laboratory for Organizational Change and Knowledge (UNLOCK). We are exploring new approaches in engaging with internal stakeholders to affect change, starting from the risk assessment and evaluation planning phase and moving towards more a facilitation role in the development of recommendations. I look forward to hearing more about developments on your side.

Add new comment

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.