IEG makes a shift in strategic focus to better support the achievement of the twin goals

The World Bank Group has set ambitious goals -€“ eradicating poverty by 2030 and boosting shared prosperity -€“ that are rippling through the development community as evident, for example, in the focus and composition of the Sustainable Development Goals (SDGs) that will be adopted this year. And it has done so amidst many challenging internal changes.

So how do we, as evaluators, take up the gauntlet of change?

Our universe is defined by the past: what the World Bank Group (WBG) has already done is what we evaluate. Typically we select sectors or themes based on the focus of the critical mass of business, or where gap analysis shows we haven't recently done work, or shows we have yet to do work at all in a given area. Eventually, new priorities and programs are absorbed and work their way through the system to become part of the evaluation universe and work plan. 

But, such is our commitment to supporting the Bank Group in helping client countries achieve its new and challenging goals, we have looked at our own processes and focus to establish how we might better and more readily contribute to success.

And that has entailed a shift to a more strategic consideration of what we evaluate. With this new focus, we are asking: What are the issues that will determine success in eradicating poverty and boosting shared prosperity for generations to come? Where are the bottlenecks that have to be addressed? What are the knowledge gaps that can be filled by evidence from independent evaluation to ensure the Bank Group, its clients, and the development community know what successes to replicate?

There are, obviously, no simple pathways to poverty eradication – if that were so, those pathways would have been identified and followed by now. The reality is that the Bank Group has to cope with significant complexity in the many environments within which it delivers a wide range of varied services in diverse sectors relating to many cross-cutting issues, and themes.

To deal with this challenge, we have decided to build part of our work program under three Strategic Engagement Areas that will help us:

  • Make strategic choices about what we evaluate to enhance our influence;
  • Build synergy between individual evaluations to deepen and integrate evidence and understanding; and
  • Create a platform for medium-term strategic engagement with Management in specific areas. 

What then are the three areas?

Economic opportunities and jobs: A key development challenge is generating growth that is inclusive, resilient and sustainable, and that creates jobs. Both the rate and quality of growth determine the impact on poverty and how prosperity is shared. This area directly connects with the ‘One World Bank Group’ agenda, including the integration of the Group’s public and private sector capabilities to advance the achievement of its strategic goals.

Sustained services for the poor: Access, quantity, quality and sustainability of services are essential for sustained poverty eradication and to ensure opportunities for the poor to live decent lives and participate in economic opportunities. While many governments in developing countries devote significant overall budget to health and education, very little may be spent on poor people. Even when public spending is targeted at poor people, resources don’t always reach frontline service providers and, where they do, quality is not always assured. 

Promoting equity and sustainability of natural capital: Natural resources, whether global commons or within national domains, are essential to creating and sustaining economic opportunities and to driving inclusion and intergenerational prosperity. The struggle for control over them is often the basis of vulnerability and conflict at local, regional and global levels. The poor are disproportionately and dramatically affected by the impact of increased scarcity of natural capital. 

The next step is to identify potentially game-changing issues for evaluation within each of these Strategic Engagement Areas, issues for which evidence from independent evaluation can help significantly shift the Bank Group'€™s ability to deliver on its strategic goals. We aim to work on addressing bottlenecks that need to be unstuck, with ideas and services that need to be fused and integrated to deliver higher, added value, and/or with the presentation of evidence that helps rethink old solutions in light of new complex problems.

So, let us know what you think the game changers are - we'™re listening. 

Comments

Submitted by Lawrence Connell on Tue, 02/17/2015 - 02:21

Permalink
The IEG’s focus in terms of issues that have critical impact on achievement of the Bank’s twin goals in a sustainable manner – economic opportunities and jobs, sustained services for the poor, and promoting equity and sustainability of natural capital – is exactly on target. Among these, the IEG’s recognition that the poor are disproportionately and dramatically affected by the impact of increased scarcity of natural capital is a critical insight that does not seem yet to have been reflected in Bank programming. The IEG might indeed want to review the Bank’s portfolio from this perspective. What efforts, beyond the WAVES Global Partnership, do Bank programs make to evaluate address the scarcity (and vulnerability) of natural capital? A second and broader approach for the IEG to fulfill its mission is directing its attention to these issues at strategic and programmatic levels. This can be achieved, for example, by evaluating the Bank’s work on Strategic Country Diagnostics and Country Partnership Frameworks—the “upstream” work that determines where and how the Bank allots its loans. Also deserving of IEG attention are the non-Investment parts of its portfolio: the Development Policy Operations and the Program for Results. To what extent are these products/programs being used to promote economic opportunities and jobs, sustained services for the poor, and equity and sustainability of natural capital? A final issue which has systemic impacts in delivery of these objectives is the formulation, implementation, and monitoring of safeguards. Since the new Bank safeguards are currently in the formulation phase, this is clearly an area which can only be evaluated over time, but it should be a high priority when the passage of time permits a review.

Submitted by Rick Scobey on Thu, 02/26/2015 - 02:06

In reply to by Lawrence Connell

Permalink
Ladd -- We appreciate your endorsement of our three proposed Strategic Engagement Areas -- and your very thoughtful suggestions on specific dimensions to explore further. We do indeed have plans to assess the treatment of environmental and social sustainability issues in the SCD/CPF process, and in instruments like DPOs and P4R -- this work is already underway. We know there is a lot internal and external focus right now on the World Bank's safeguards policy framework, and we are reflecting how we can best bring IEG evaluation evidence into the discussion. Stay tuned -- and please continue to provide feedback -- we greatly value the perspectives and experience of CSO organizations and other external stakeholders.

Submitted by Tessie Catsambas on Tue, 02/17/2015 - 22:30

Permalink
This is an excellent framework for asking evaluation questions around the equity and shared prosperity goals of the Bank, enhanced by Lawrence Connell's useful comments. First, I have a point of clarification: is this framework consistent with the design of new programs at the Bank? In other words, can the IEG unilaterally decide on the three engagement areas, or do these need to be reflected in the Bank's work programs? Will the IEG use this framework(once finalized) consistently in every evaluation? As I reflect on enablers to the game changers suggested, I cannot help but think about country ownership and local leadership. So, in every evaluation, I would ask, "In what ways is this project promoting and strengthening country leadership of the twin goals?"

Submitted by Rick Scobey on Fri, 02/27/2015 - 05:42

Permalink
Tessie -- Excellent questions, as always! First -- we selected the SEAs in light of the strategic directions of the new WBG Strategy and the "volume" of WBG assistance, in line with our mandate to effectively assess the relevance, efficacy, and efficiency of WBG operational programs and activities, and their contribution to development effectiveness. We also factored in the emerging global consensus on the SDGs, as part of our reflection on whether there are any "blindspots" in the current focus of WBG support. By design, we wanted to focus on broad thematic areas that cut across the work of several of the WBG Global Practice and Cross Cutting Solution Areas, so that we don't end up doing evaluations in "silos." Second, the SEAs are more about selectivity, focus, and alignment of our work (not just major evaluations but also learning products and project-level evaluations), rather than a methodological framework. Our intention is to apply a core methodological framework across all the SEAs and all evaluations, which emphasizes the key factors to enhance WBG contribution to the twin goals: gender and social inclusion, distributional effects, environmental sustainability, capacity development (as you rightly emphasize), etc. Our growing focus on "game changer" evaluations is both ambitious and challenging -- we will clearly need to reflect, learn, and recalibrate as we move forward! So please keep asking the tough questions!

Submitted by Tessie Catsambas on Thu, 02/26/2015 - 22:46

In reply to by Rick Scobey

Permalink
Rick, thank you for your informative answer. I am very excited about the vision of evaluation in line with the SDGs and the twin goals, as they are likely to be interesting for a broader audience. This makes ask a further question. As the IEG launches into these exciting evaluations, how will you keep Bank staff, who are not involved in the specific program being evaluated, abreast of lessons and observations? In other words, how will you maximize evaluation use beyond the bilateral discussion with a program? Maybe even to audiences outside the Bank...

Submitted by Krishna Belbase on Thu, 02/26/2015 - 22:30

Permalink
"Game changer evaluations" sounds exciting. It would be more exciting to add an “impact focus” to it as that is what matters the most when it comes to assessing results (as well as learning) from big investments that the Bank Group and its allies invest in. One could also argue that what really matters (for changing games meaningfully) is the focus on results including related equity and environmental sustainability dimensions. So… if we assume that what is needed is "game changer impact evaluations"… the next question that comes to mind is whether the present theories and practice of impact evaluation would fit that purpose? That is a big question that need to be answered collectively by the evaluation experts and the key stakeholders (particularly the donor community). Impact evaluations require data and expertise which require financial resources and time that are not always affordable. Perhaps what is needed is identifying and using low cost and rapid evaluations that are impact focused and provide answers to key game changing questions within the timeframe when they need to be answered.

Submitted by Rick Scobey on Tue, 03/03/2015 - 05:39

In reply to by Krishna Belbase

Permalink
Hi Krishna -- Excellent point -- it is indeed all about results! And we share your view that (1) impact evaluations are one important tool to assess impact, and (2) we need to focus on mixed methods to build a robust and cost-effective framework to report on the results and performance of our institutions. Do you have any helpful experience to share at UNICEF about low cost and rapid evaluations to assess results on the ground?

Submitted by Krishna Belbase on Fri, 05/22/2015 - 06:49

In reply to by Krishna Belbase

Permalink
... Sorry about a delayed posting. I came across this report of a nutrition survey which used SMART method. The survey was done in 5 weeks during Feb-March in 2014 in Typhoon affected regions in Haiti. https://www.humanitarianresponse.info/system/files/documents/files/SMART%20Survey%20Final%20Report.pdf Curious to know where this important discussion has converged.

Submitted by Joe Ryan on Fri, 02/27/2015 - 22:32

Permalink
Good job of narrowing your focus. However, you might want to expand your definition of natural capital by going beyond the rather simplistic term ´natural resources´, and highlight the broader term ´ecosystem services´ (ES), Besides their Production Function (e.g., producing natural resources, including water in arid areas), they include several of nature´s other functions , including the capacity to assimilate air, water and terrestrial pollutants and especially to reduce the poor´s high level of vulnerability to climate change impacts (e.g., mangrove forests as buffers against powerful storms or in reducing erosion). However, Incorporating resilience could become a cross-cutting parameter issue in each thematic area (e.g., changes in economic, social service and ecosystem resilience) and addressed in future evaluations, provided that the evaluation indicators are SMART (in order to ensure that the term resilience simply become another metaphorical panacea) . Besides its practical utility (dynamizing the three dimensions of sustainable development), resilience could also contribute to create a way that your evaluations could also become more dynamic by promoting adaptive, learning-based governance systems to tackle the multi-dimensional soluitons that are required to confront poverty.

Submitted by Rick Scobey on Tue, 03/03/2015 - 05:48

In reply to by Joe Ryan

Permalink
Joe -- Extremely helpful and thoughtful comments -- many thanks. We do intend to look at broader environmental services, so your point on being careful about a narrow definition of natural capital is spot on. And your point on treating resilience as a cross-cutting issue across the three SEAs makes a lot of sense. One challenge -- we will need to strengthen our internal skills and methods for robust assessment of resilience. Any thoughts on where we should be looking for examples of good practices and strong methodological frameworks?

Submitted by Waqar Ayub on Sun, 03/01/2015 - 03:56

Permalink
World Bank's lending policy mostly drive the borrower to come up with programs that will easily sail through the Board when presented and bring in the much needed resources. In Southeast Asia, focus has been at improving social services at the cost of job creation, due to which the MDG achievements have been far from the agreed targets. People have fallen in the poverty trap because of minimal earnings which did not allow them to provide for healthcare and education. Borrowers too have been coming up with figures to ensure that the next tranche comes in thus tweak monitoring and evaluation results to suit their needs or make promises to achieve results by next evaluation. Once Bank policy starts tying up social projects with job creation/increased earnings, borrowers will be more open and see multilateral not as selling their projects, but really concerned about borrower's well being. When borrowers start sharing bottlenecks/problems, monitoring and evaluation takes a totally new dimension i.e. a two way traffic and transforms into a tool leading to realizing of objectives.

Submitted by Rick Scobey on Tue, 03/03/2015 - 05:59

In reply to by Waqar Ayub

Permalink
Waqar -- Great to have the "frontline" perspective from the country-level. Your comments underscore the importance of building meaningful beneficiary engagement into our evaluation work, so that we can access citizens' views on the relevance and effectiveness of WBG country assistance strategies and related investment operations. We are launching an evaluation this year that will review the initial implementation experience with the WBG's new "country engagement model," which is supposed to address the kind of issues you are raising. Please stay tuned, and come back with feedback as we share the findings.

Submitted by Rick Scobey on Tue, 03/03/2015 - 05:30

Permalink
Tessie -- We are indeed trying to ramp up knowledge sharing and communication about our evaluation work, both internally within the WBG and externally to practioners, country clients, and other stakeholders. Our recent client survey (see today's blog) highlighted that we still have a ways to go to get clients to read and use our findings. This blog is one example of trying to reach out to a wider audience -- other steps include: producing shorter and more accessible reports that people will actually read; making our internal and external websites more user friendly; producing more "bite sized" summaries of lessons learned and good practices, in a format/platform that people can access easily and when they need the knowledge; and investing in more innovative learning events and tools, including social media, that fit more with how adults actually access knowledge and learn. All of these steps need to be accompanied with a focus on IEG's own behavior and incentives -- we need to deepen our own internal culture that learning and dissemination of evaluation results is as important as conducting high quality evaluation work!

Add new comment

By submitting this form, you accept the Mollom privacy policy.