Faced with growing demands for evidence-based decision making and measurable results, do we need to look for new ways to evaluate the evaluators?

What works and what doesn’t? It is a simple question and one that drives the work of IEG in evaluating the effectiveness of World Bank Group performance.

Through our evaluations we provide evidence on results and performance of Bank Group interventions and learning on how to repeat success and avoid future failures. But how do we measure our own success? Are we able to use similarly rigorous methods to determine whether IEG is having the desired impact?

The answer to that last question is – not yet.

Preparing for impact

Influence is difficult to measure, but it is at the heart of what we seek to achieve. And, it starts with the choices we make about what we evaluate, and how we undertake our work. As with any development intervention, we need to ask ourselves from the outset: which processes, which choices do we aim to influence? For each evaluation, we ask ourselves who are the stakeholders and what are the policies or decision-making processes that should benefit from the evidence we generate. Being clear now about which processes, discussions, and decisions we want to influence helps us align our work program and channel our resources and efforts towards those outcomes. This is why we need a results framework for IEG.

Being effective means being heard – and knowing it!

In addition to planning for impact, we need to ask ourselves: are our independent evaluations adequately informing decision-making, improving Bank Group outcomes and adding to the body of knowledge on how best to get results? 

Through our discussions with the Board and our interactions with Bank Group staff, we know that most of our evaluations are well-received. One measurement tool that we currently use is an annual client survey. This gives us a good indication of how well, or not, we met client needs. The results of the last survey, a year ago, showed declining trends in perceptions of our work. Does this mean we are losing influence?

I believe strongly that IEG contributes to the evidence-base that helps Board and Management take better informed decisions, which in turn improves World Bank Group performance and results. We now need to be able to provide the evidence that this is so.

Tracking results from the outset

At a recent IEG retreat I challenged staff to develop a new framework that allows us to define our objectives and better track our performance and results.  The discussions took place around a number of topics and produced inputs that we are now translating into our upcoming IEG Results Framework for inclusion in our work program.

Here’s a preview of the direction in which we are going:

• Our goal is to help accelerate the development effectiveness of the Bank Group. With the twin goals this has become measurable and makes it easier to identify how we can, with independent evaluation evidence, help achieve these goals.
   
• Our objectives in the medium term are to shed more light on what works and what doesn’t in inclusive growth – from poverty eradication, to shared prosperity, gender equality and sustainability.
   
• For our next three year cycle – from mid-2015 to mid-2017 – we will focus on evaluating the:
  • Implementation of the Bank Group strategy in select areas to inform Board and Management about early corrections, if needed; and
  • Performance and results in low income countries funded by the Bank’s International Development Agency (IDA) to inform the next replenishment discussions of IDA Deputies.

If that is what we want to achieve, how would you measure it? How to evaluate the evaluators? In many ways it is a more difficult question than “what works”, but it is one that is just as important to answer.

Your thoughts?