Regular readers of this blog know that in IEG we are committed to a strategy that helps us “do the right things,” “do them right” and share our knowledge in cost-effective ways.

So how do we in IEG make strategic choices? We have four ingredients:

  • A Gap Analysis to assess evaluation coverage over past years, related to the size of the World Bank Group;
     
  • A new Results Framework to identify where evaluation evidence is needed and how it will influence Bank development effectiveness;
     
  • Stakeholder consultations within the Bank Group, both Board and Management, and externally including think tanks and CSOs to solicit different perspectives on the top evaluation priorities; and
     
  • Selectivity Sheets prepared for all proposed topics to assess relevance, potential influence, and IEG readiness and value added. These are attached to our work program for you to review.
     

We have a rolling three-year work program. Each year we take a look at it, determine whether our priorities are still right, revise it as needed, and add new ideas for the outer years of the program.

For the next planning cycle, we will be doing a couple of things:

  • Making sure through stakeholder consultations, especially with colleagues in the Global Practices and Cross-Cutting Solutions Areas, that we clearly define our division of labor and expectations of each other. We’ve already had a number of requests for services to share lessons from our independent evaluations. It’s a great start that we want to maintain through our consultation process. At the same time, we will continue asking other stakeholders for their views as well.
     
  • Update the gap analysis to reflect the structural changes in the Bank Group. This is a complex undertaking as the range of Bank services is far reaching and we need to take all of them into account, including account knowledge work and its convening role, as much as lending, investments, and guarantees.
     
  • The new results framework challenges us to identify where evaluations can make the biggest difference in enhancing Bank Group development effectiveness in attaining the twin goals of poverty eradication and boosting shared prosperity. For me, this means identifying the key drivers of development effectiveness by using evidence from independent evaluation to unravel bottlenecks. Not an easy task for a complex institution like the World Bank Group. 
     
  • Unpacking the question of the adequacy of our evaluation budget. Are we over-evaluating or under-evaluating the Bank? If we want to affect Bank Group development effectiveness could we achieve more if there were more resources for independent evaluation, or have we reached a healthy limit? Is the supply of independent evaluation beyond the Bank’s absorptive capacity, or is the need for independent evidence such that we should do more?

In short, the groundwork we laid during last year created a great platform for this year’s work program and for new challenges in the years to come. 

Comments

Submitted by Isaac .O. Otieno on Tue, 09/02/2014 - 22:54

Permalink
As an evaluator am interested in your content.

Submitted by Caroline Heider on Wed, 09/03/2014 - 05:35

In reply to by Isaac .O. Otieno

Permalink
Many thanks, Isaac, for your interest in our blog. I hope it is useful and look forward to comments and suggestions from you and other readers.

Submitted by kassem on Wed, 09/03/2014 - 07:56

Permalink
It would of great value in evaluating the Bank's projects and programs not to forget the full picture and ensure the link between them and the overall strategy. Projects are not executed in vaccum. Their evaluations need to verify/ validate their alignment with the overall strategy/ plan.

Submitted by Geeta Batra on Thu, 09/04/2014 - 23:13

In reply to by kassem

Permalink
Thanks, Kassem, for your comment. We recognize that projects are linked to the broader country goals and strategy. When we review country strategy completion reports, these different dimensions come together and the link between projects, sectors and the broader strategy is reviewed. At the project level, the strategic relevance is validated, which includes the alignment to the overall strategy.

Submitted by Daniel Bassey on Fri, 09/05/2014 - 23:39

Permalink
I still believe that procurement experts have a major role to play in the achieving of any success in project execution. Procurement is the channel through wish a good intention remains good or goes bad. An evaluation on accessing the fairness of quoted budgets matters a whole lot and effort should be made to have the right persons execute projects so that the expected result is met conveniently.

Submitted by Charles Lor on Sun, 09/07/2014 - 04:57

Permalink
To meet accountability needs means identifying the key commitments made and the work program would be relatively invariant and focus on completion evaluation. New priorities would be added with new programs and commitments. Lesson learning would be limited because programming would have shifted already and new generation phases would have been launched already. For the purpose of lesson learning, to be relevant IEG would have to engage in midterm reviews and have a forward looking selection. Completed programs that reflect another generation of programming (favored under the accountability selection) would be discarded. The focus would shift to emerging topics and evaluating pilot initiatives. How does IEG select according to these two conflicting mandates?

Submitted by Caroline Heider on Mon, 09/08/2014 - 02:36

In reply to by Charles Lor

Permalink
Charles, good points, just as in your last contribution on VfM. You raise questions that accompany us, as evaluators in general (not just in IEG) all the time: if we come in earlier, there might be more learning, but we cannot assess results and it would impede our impartiality for later evaluation. As you would know, in the WBG we are fortunate to have a multi-layered evaluation system, that includes self-evaluation by Management where mid-term evaluations take place. At the level of country engagements, we are in the process of looking at our role in these exercises similar to the validation we do for projects and country strategies at the end point. But even then, this would be possible only on a select basis. For projects, it is really the accountability of Management to undertake mid-term evaluations and take corrective actions, something President Jim Kim is very keen to seen, while we will continue to ensure results reported by Management are validated and independently evaluated to assess outcomes that would not have materialized during implementation. And, to keep contributing the latest agenda, just as we did with recent evaluations of, for instance, PPPs, Health Finance, or Procurement, we conduct major evaluations that cover a 10 year period (including up to the year when the evaluation takes place) and are timed to feed into corporate processes to help learning from the past for the future direction of the WBG. It is this combination of different evaluation tools that helps us play a range of roles, while maintaining the independence that is essential for our mandate.

Submitted by Anjali Kumar on Thu, 09/11/2014 - 02:02

Permalink
Daniel, Indeed you are correct to point out that procurement modalities have a major role to play in determining the success of project execution. This is one reason why IEG has just undertaken a major evaluation of the World Bank's procurement system. Good procurement goes beyond fair budgets for goods or services needed for a project. It means achieving the best value for money, in an efficient and transparent manner, and with fair opportunities for all suppliers to participate. You can find the complete IEG evaluation at http://ieg.worldbankgroup.org/evaluations/world-bank-group-and-public-procurement

Add new comment