The World Bank Group is engaging in an exciting change in culture.  Solutions will trump what President Jim Kim calls “mosquito bite projects” and the focus will be squarely on results to help client countries achieve the ambitious dual goals of eradicating poverty and boosting shared prosperity.

Changing culture is always hard, and the new WBG Strategy rightly focuses on the critical role of incentives to motivate staff to act differently.   

IEG clearly plays a role in setting incentives for operational staff – but are we encouraging increased focus on results, risk aversion, or “bean counting”?  Knowing how our metrics – what we evaluate and how – influence people’s behavior, we have taken a close look at the World Bank Group Strategy to ensure that our work program and our own behaviors clearly support the change process. 

Here are four ways in which we will do this:

Role Modeling is a powerful way to demonstrate different behaviors.  So, in our current work program, we have committed to review and sharpen our own results framework to clarify what we want to achieve and how we will measure success.  We intend to hold ourselves to the same standards as we do WBG operational staff, and ensure transparency and accountability in reporting how well we deliver impact.

Creating Demand for Outcome Data. In our evaluations, we aim to increase our focus on results, the changes in the lives of people affected by WBG interventions.  We will do so using and bringing together data that already exists -- the M&E data collected through projects, impact evaluations, “Big Data,” qualitative evaluations, to name just a few -- and combine this wealth of data with outcome information collected through our independent evaluations.  By showcasing what we are looking for in terms of results and how to use data to track and demonstrate results, we aim to provide examples that can be followed by others,  and incentivize effective collection, analysis and use of results data.

Empower People with Information.  Seeing improvements in the lives of people is a strong driver for WBG staff. They will be even more motivated by information – evidence from evaluation – that helps them understand what difference they make, and why or how these results come about.  As independent evaluators, we have a couple of comparative advantages in this respect: it is our job to look back, bring together data and information from various sources, and help create an understanding of what works, what doesn’t and why.  In addition, our independence gives us the opportunity – even obligation – to face the hard facts without which change will not happen.

Recognition of Excellence.  IEG Awards have been in place for many years to reward individuals and teams in the World Bank Group for outstanding achievements.  We are now redesigning the Awards to better recognize the behaviors that the Strategy seeks to promote –candor and learning from mistakes, collaborating across the three Institutions and with others, using evidence in decision making, and exercising selectivity and making tough choices.  We will shine the spotlight on teams that have succeeded and can serve as a role model and help others.

Changing culture is everyone’s business and we at IEG are prepared to play our role.

Comments

Submitted by Milo Stevanovi… on Tue, 11/12/2013 - 01:35

Permalink
Our projects operate in rapidly changing environments, particularly IC in the MENA region. Changes come daily -- if not momentarily. To deliver real reform with maximum impact we need the agility to adapt quickly -- to both traverse obstacles and to seize opportunities that present themselves as we work, to realign resources, methodologies -- even products -- to effectively respond to evolving client needs. Currently, project offerings and M+E are relatively static: project activities (and corresponding targets, outputs, outcomes...) are set long in advance, changeable only at certain intervals. (Some have commented that this is the functional equivalent of permitting steering only once or twice on a year long car journey.) Will the new structure focus more on accountability for the ultimate reform impact (rather than on measurement of all the detailed steps along the way? One would think that it will need to, how else will the structure accommodate the experimentation, innovation, collaboration and learning-from-mistakes that is contemplated? In fact, I would go further to say that we should develop a structure that encourages and enables a much increased failure-learning iteration rate. We should not have to wait until the end of an entire project to learn and benefit from our lessons -- we should really be learning and adjusting all the time. We welcome the changes contemplated and applaud efforts in this regard.

Submitted by Caroline Heider on Wed, 11/13/2013 - 02:37

Permalink
Milo, thank you for your thoughtful comment. In addition to structures, behaviors will be very important as well. We are evaluating how learning happens in World Bank lending to she more light on this aspect of project implementation, including what conditions are conducive and which aren't. It is something very new and we hope that the lessons -- both for the Bank and for us for conducting this type of evaluations will take us a big step forward.

Submitted by Joy de Beyer on Mon, 11/25/2013 - 22:50

Permalink
Dear Caroline, Ive long thought that IEG is in many ways a huge missed opportunity to really contribute to Bank learning. The ICR process is not conducive to really learning from project experience. The ratings feel rather mean-spirited - the criteria for the highest rating are almost impossible to meet. And to ask "Did the project set out to do what it said it would do 5 or more years ago" isnt necessarily the most useful question. I understand the rationale from an accountability viewpoint, but too often the ICR seems less about really learning and more about avoiding an acrimonious exchange between the region and IEG. Perhaps we are all too "Type A" to cope with less than A grades on anything we do, that is a key barrier. But its exciting to read your blog and determination that IEG do much more to help identify solutions

Submitted by Caroline Heider on Wed, 11/27/2013 - 05:05

Permalink
Joy, I really appreciate your honest feedback, which is probably shared by other operational staff. One of the important goals of the ongoing WBG reform agenda is to accelerate and deepen operational learning and TTL reflection during project implementation, so that teams focus more on course-corrections when they are needed, which should reflect positively on project results at completion. As follow up, IEG and WBG Management have begun to discuss improvements to the ICRs and their reviews to capture better project results and what can be learned from them. Our goal is to deepen learning, facilitate monitoring and reporting of results to feed into a corporate scorecard, and ensure accountability for WBG delivery of results against agreed objectives. Building an effective approach to assessing and validating project completion is clearly a critical step in delivering the new "Solutions Bank". And while there is need and commitment to review the ICRs and their reviews, I think the real drivers for change will be incentives and behaviors -- for both operational teams and IEG evaluation teams.

Add new comment