Results of IEG's client survey – an exercise in evaluating the evaluator.

As evaluators we have to "walk the walk" and hold ourselves to the same standards we expect of others. In my blog When Rating Performance Start with Yourself, I spoke about the importance of being heard and knowing it, that is, the importance of feedback. One part of the evidence trail that tells us what difference we make is feedback from the users of our evaluations.

Late in 2014, we asked our clients at the World Bank Group (including the Board, management, and staff) and outside stakeholders (including academics, representatives of governments, international organizations and NGOs) to tell us what they think of our work.

Thanks to the many of you who responded! We had much greater engagement and higher response rates than in any previous survey of this kind. And here'€™s what you had to say:

How relevant is IEG to the mission of the World Bank Group mission?   

• Board members, who showed a high level of appreciation in the previous survey, increased their assessment of our relevance in the top response category from 26% to 40%. Taking all ratings "above the line", we are hitting 100%.
   
• Colleagues at the World Bank Group are not rating us as highly, but their assessment of our relevance has stayed stable at around 88%, in the range of highly to somewhat relevant. 
   
• Clients outside the World Bank Group give a high rating (95%) for relevance. Respondents based in Africa, who showed a strong level of participation in the survey, were also the group of external clients who thought our work was most relevant.

Are we getting the balance right between accountability and learning?

• Board members and external stakeholders are more satisfied than staff with the balance we achieve between accountability and learning in our work. Staff, particularly senior staff and those most familiar with IEG's work, believe we are overly focused on the accountability remit at the expense of learning.

How good are our evaluations?

This is an area of high importance where we have already, and will continue to invest a lot of effort:

• Feedback on the quality of our work shows that on a scale of 1-6, with 6 being the highest, we have come out with an average of 4.2. Yes, well above the mid-point of 3, but not good enough. The client survey provided us with feedback on the overall quality, clarity, coherence, feasibility, and cost-effectiveness of recommendations. We are looking into these details to understand how we can do better.

• The Survey also tells us that we need to be paying more attention to the timeliness of our work. Satisfaction levels among Board and WBG staff respondents have decreased with reference to this important criterion; and it is the one we rate lowest on when it comes to the evaluation process.

Aiming for the highest quality has been the main reason for falling behind on timeliness, so we are looking to address both of these points together to find ways to improve quality and efficiency.

To what extent are our products being used?

• We have some way to go in this area too. Among World Bank Group staff responding to our survey, 65% said they are using our evaluations for various purposes. While the usage has gone up across all categories - whether in the form of advising clients, to designing results frameworks, projects or policies - we want that percentage to be much higher.

• Among respondents from the Board, the percentage reaches above 90% for the use of our work in assessing country strategies, and 75% and above for reviewing projects, policies and procedures, sector strategies, giving inputs to the work of others or proposing a course of action.

• A question that remains for us: if our authorizing environment, the Board, is such an avid user of insights from independent evaluation, how do we get others to take those lessons on board earlier and more readily in the process?

Are we making a difference?

Here the news is really positive. All categories of respondent have rated our impact and influence on the World Bank Group and the broader development community much higher than in the previous survey.

• 94% of respondents from the Board find that IEG's work influences the World Bank Group's development effectiveness at least to some extent. For external audiences, this rate is 90%, and 75% of colleagues from the World Bank Group think so too.
   
• We see a similar pattern, though at a slightly lower level, for IEG'€™s influence on the broader development community: 88% of Board members, 83% of external clients, and 63% of World Bank Group staff see IEG having at least some influence, often more than that.

How well do we communicate with you?

• Email announcements, our website, as well as events and presentations provide the main access route to our product, particularly for external stakeholders who also appreciate our growing social media outreach efforts.

And where do we go from here?

The client survey is an important part of a range of structured feedback that we are about to receive regarding what works at and for IEG. We are currently discussing the best ways to respond to your feedback so that, next year, we'€™ll be even better.

You can access the full set of results here.