Displaying 1 - 10 of 3042
What works in public utility reform: Lessons from evaluations in the energy and water sectors
Web Resource
Utility reform has never been more important. The COVID-19 pandemic has badly impacted utilities across the world. Many utilities are now under intensified financial stress due to budget reductions and a loss of revenue, resulting from a sudden drop in collection rates, suspension of billing, and tariff adjustment in some countries. This, in turn, has made it more challenging to ensure continued Show MoreUtility reform has never been more important. The COVID-19 pandemic has badly impacted utilities across the world. Many utilities are now under intensified financial stress due to budget reductions and a loss of revenue, resulting from a sudden drop in collection rates, suspension of billing, and tariff adjustment in some countries. This, in turn, has made it more challenging to ensure continued service delivery.
IEG recently published the synthesis Public Utility Reform: What lessons can we learn from IEG evaluations in the energy and water sectors?, a compilation of evidence of what worked and what did not work, and why, in World Bank support of public utility reforms in the energy and water sectors in its client countries. Its findings are even more relevant in the context of uncertainty about medium-to long-term outlook for recovery from the challenges imposed by COVID-19.
Well before COVID-19, financial viability and institutional accountability were the two main challenges faced by public utilities in improving service outcomes in the energy and water sectors. Now, the effectiveness of utilities in these two fundamental areas remain critical for ensuring the quality and sustainability of these vital basic services during a post-pandemic recovery.
Financial Viability
IEG analysis reveals a range of World Bank interventions geared to support financial viability in both the energy and water sectors.
Recovering the cost of service is at the core of sector reform. Across both water and energy sectors, inadequate cost recovery is a key driver of financial underperformance. Poor bill collection and operational inefficiencies (including excessive network losses) also have a significant role. IEG finds that, while tariff reform is fundamental, improving operational efficiency of service providers is crucial for financial sustainability. The cumulative evidence indicates that when inefficiencies result in high-cost service provision, improving utilities’ operational efficiency should precede or go hand-in-hand with tariff increases. Additionally, the gains from reductions in technical and commercial losses, improvements in payment collection, financial management, and demand side management proved easier to sustain once implemented.
Evidence points to the importance of strengthening utilities’ commercial orientation, which is vital for the provision of adequate and reliable services, regardless of whether the service delivery agents are under public or private ownership. Utilities that emphasize cost control, customer orientation, and responsiveness to incentives are more likely to make meaningful progress. For example, World Bank operations in Vietnam and Turkey helped improve financial sustainability of electric power utilities through technical support and policy reforms, incrementally implementing tariff and market regulations in the electricity sector.
Utilities may need more financial support as they weather the economic crisis triggered by the pandemic. However, as the recently published IEG evaluation State your Business! An evaluation of World Bank Group support to the Reform of State-Owned Enterprises cautions, temporary subsidies introduced at the time of COVID-19 can pose “policy traps” supported by powerful vested interests, which can be hard to reverse once the crisis is over.
Institutional Accountability
Creating the right accountability and incentives is essential for effective service delivery. In both energy and water sectors, institutional accountability is critically tied to performance. Sustaining reforms requires competent institutions and strong administrative capacity. Improved performance can be a first step towards attracting private sector investment.
Strengthening sector planning, utility management, capacity and skills, can improve sector outcomes. A solid sectoral fiscal, financial, and regulatory framework also defines and sets the context for leveraging markets and the private sector to support service delivery.
There are multiple institutional pathways that could lay the foundation for improved and sustained service delivery. There is no single model but there are certain principles that work.
In energy, improved accountability and regulatory performance drive sector outcomes. Good practices on corporate governance and regulation enable the sector environment to leverage markets and the private sector. In Rwanda, for example, the World Bank (through budget support operations), the International Finance Corporation (through advisory services), and Public-Private Infrastructure Advisory Facility (PPIAF) helped the government develop sector regulatory structures and separate water and electricity utilities to improve governance, accountability and transparency. Institutional and policy reforms transformed the Rwanda Energy Group into a commercially operated state-owned enterprise and helped attract private finance.
In water, improved capacity, incentives, and transparent rules on accessing funds can ensure good sector outcomes. Good financial and operational data systems are also important. In Peru, the utility Sedapal radically changed its corporate management approach and work culture, including adopting a new performance-based compensation and incentive system driven by reaching results targets. IEG’s field-based assessment confirmed a steady improvement in access coverage, basic service parameters, and operational and financial performance.
Political and social challenges
In both sectors, utilities' operations and management are closely linked to the political economy in which they operate. Political economy considerations can inform specific design elements, including choices of programmatic instruments vs. standalone operations, or front-loading vs. back-loading of important reform actions in a programmatic series. Experience shows that support to operations need to match the time frame in which effective government action can reasonably take place. The World Bank’s experience shows that complementary interventions and sustained support contribute positively to favorable and enduring results.
Regarding tariff reform, the institutional, political, and social challenges are considerable. Public opposition to tariff reforms reflects a lack of confidence in public service improvements and that vulnerable groups will be protected. At the same time, it is important to address potential negative distribution consequences of reforms through such measures as differentiated tariffs and targeted assistance programs. Their success depends on the government’s ability to reach vulnerable households through fiscally sustainable programs.
Read the report | Public Utility Reform: What lessons can we learn from IEG evaluations in the energy and water sectors?
Pictured at top, clockwise from left:
The main drinking water pipeline for 750 households in Alapars and Karenis communities (Kotayk region) being fully rehabilitated. Armenia. Photo credit: Armine Grigoryan / World Bank
The control room at the thermal power station at Takoradi, Ghana, June 21, 2006. Photo credit: Jonathan Ernst/World Bank
Electricity Transfer Station in Kenya. Photo credit: Andrew Stone
Windmill, Nicaragua photo credit: Ihsan Kaler Hurcan
Wegala Community Water Supply and Sanitation Project. Sri Lanka. Photo credit: Simone D. McCourtie / World Bank
Girl gathers drinking water from a community water pipe. Photo credit: Dominic Sansoni / World Bank
State Your Business: What are the keys to successful reform of state-owned enterprises?
Web Resource
State-Owned Enterprises SOEs are critical to many developing and emerging economies where the lives of millions of citizens are deeply affected by how these enterprises are run. Governments use SOEs to provide services across multiple sectors and to address the impacts of economic downturns or crises, such as the current COVID-19 crisis. Show MoreState-Owned Enterprises SOEs are critical to many developing and emerging economies where the lives of millions of citizens are deeply affected by how these enterprises are run. Governments use SOEs to provide services across multiple sectors and to address the impacts of economic downturns or crises, such as the current COVID-19 crisis.
Although many SOEs are run well and have made important economic contributions, many others suffer from low productivity and efficiency, which have a detrimental impact on growth and consumer access to services. SOEs’ mixed institutional mandates and their political importance often pose performance and governance challenges. Poor performance can also generate substantial public fiscal losses. Aware of the importance of SOE reform to achieving economic development and service delivery goals, the World Bank Group (WBG) has long supported developing countries to address the associated challenges.
IEG recently published the evaluation State Your Business! An Evaluation of World Bank Group Support to the Reform of State-Owned Enterprises, its first systematic assessment of the Bank Group’s support for the reform of SOEs, focusing on the energy and financial sectors, where Bank Group investments in the last decade have surpassed USD 70 billion.
The political importance of SOEs can impose substantial challenges to introducing and sustaining reforms. Such political economy risk factors help to explain why countries such as Bangladesh, Egypt, and Indonesia have signaled their intent to privatize state-owned banks but later halted efforts because of internal political constraints. The allocation and pricing of power and finance can evoke intense public reaction and mobilize vested interests. Further, temporary crisis response programs can turn into irreversible “policy traps”, locking SOEs in to underfunded mandates even if it damages their long-term viability.
IEG’s evaluation sheds light on key factors driving successful SOE reforms and points to risks and obstacles that limit reforms with potentially dire consequences for SOE performance and delivery of services to the public. These factors include:
Sector competition
Research on the subject shows that enhanced competition improves SOE performance in both the financial and power sectors, both by itself and in combination with other reforms. First, there is strong evidence that SOEs perform better in the power sector (and in general) when competitive conditions prevail at the sector and enterprise levels. For example, an econometric assessment of power sector data for 36 developing and transition countries over 18 years found that gains in economic performance stemmed mainly from allowing private participation.
Privatization or regulatory reforms were less effective without a competitive market. Private sector participation can take various forms and involve different functions of the power market (generation, transmission, distribution, and retail). A key aim of Bank Group support has been to strengthen competition and regulation in SOE markets, in part to foster a level playing field between SOEs and private companies
One example of WBG support to achieve a more competitive power market is in Vietnam, where Electricity of Vietnam (EVN) and other SOEs dominated power generation. EVN also fully owned the entity that operates and maintains the national transmission grid.
The Bank Group engaged comprehensively in all aspects of the power sector (rural electrification, generation, transmission, distribution, load dispatch, renewables, development of wholesale and retail power markets, regulatory aspects, and SOE reform), using a wide range of instruments. The credibility and trust generated enabled the Bank Group to support the government in sequencing sector wide reform.
In 2012, EVN unbundled its generation subsidiaries into three separate generation companies, at the same time as the launch of Vietnam’s competitive generation market, in which independent power producers and generation companies compete in a power pool to sell to individual buyers. As of 2016, 24 percent of installed capacity in Vietnamese power generation was privately held.
In the financial sector, research shows bank concentration is more constraining to firms’ access to credit in countries with higher shares of state bank ownership. More competitive environments enhance the benefits of bank privatization.
The Bank Group's private sector arm, the International Finance Corporation (IFC) has committed to promoting “competitive neutrality” in the SOEs that it invests in.
Given the greater success of reforms with better competitive conditions, the IEG report recommends that the World Bank Group should gear up to do more competition analysis at the sector and project level, as part of a selectivity framework for engaging in SOE reform.
Control of Corruption
The oversight and accountability challenges of SOEs noted above can make them hard to manage and frequently exposes them to corruption. Corruption powerfully undermines SOE performance. In Ukraine, for example, widespread corruption impeded the progress of SOE reforms. By June 2018, more than 194 of the 793 criminal proceedings handled by Ukraine’s National Anticorruption Bureau dealt with about 50 SOEs and their officials, according to an OECD report. In Kenya, petty corruption among field staff responsible for installing and reading electricity meters reportedly hindered efforts to stem power system losses.
The IEG report finds that a country’s control of corruption is strongly associated with the likelihood of SOE reform success. Other things being equal, a country with high control of corruption is more than twice as likely to see SOE reform interventions succeed as a country with low control of corruption. In conditions of low control of corruption, it is more difficult to strengthen the governance, regulation, or performance of public enterprises.
IEG thus recommends that the Bank Group apply a selectivity framework for its engagements on SOE reform that considers country governance conditions. Where corruption control in the country is weak, IEG recommends that World Bank Group either sequence SOE reforms (first addressing public governance) or actively mitigate corruption risks through close attention to the strength of client commitment, supervision, simplicity of project design, and appropriate sequencing.
Mobilizing Private Finance
In its review of the Bank Group's experience, IEG found positive experiences from collaboration across its institutions, a key to mobilizing private financing and capabilities. Yet, IEG found these examples somewhat infrequent. The IEG report recommends scaling up the collaborative approach known as “Maximizing Finance for Development” (MFD) to enhance internal coordination among WBG units and help mobilize private financing and capacity including through ownership reforms such as privatization and public-private partnerships.
However, the recommendation to prioritize private solutions for SOE reforms through an MFD “cascade” approach is not a call for privatization alone. Rather, the World Bank Group should consider a full range of options, including improving regulation and competition, strengthening SOEs’ corporate governance, or supporting ownership reform. The end goal may range from better preparing SOEs to tap private capital markets, to creating a level playing field for competition between private companies and SOEs, to public-private partnerships or outright privatization.
Sequential and Complementary Support
IEG found that sequential and complementary interventions, often involving more than one Bank Group institution, aid successful reform. This includes good prior analytic work. For example, over many years, WBG strategies and programs in Bangladesh’s power sector were aligned with successive government five-year plans. Since at least 2004, the Bank Group engaged in unbundling and building technical capacity through financing and technical assistance. This covered regulation, generation, transmission, and distribution.
The World Bank also supported the successful Power Cell, which channeled technical, planning, and coordination support to government while facilitating the role of private power producers. The regulator, Bangladesh Energy Regulatory Commission, benefited from WBG support since its creation. The World Bank, IFC, and MIGA (the WBG’s agency for insuring against political risk) were all involved in a “cascade” approach in supporting independent power providers. Over time, sector performance improved in reduced losses, reduced arrears, and an elimination of the energy gap, with the Bank Group as a trusted partner bringing expertise in the field, access to global expertise, long-standing relationships with key government agencies, coordination of donors, and a consistent policy view.
Through selectivity, coordination and sequencing, the Bank Group can help client countries better serve their citizens through SOE reform. This is especially pressing now as governments cope with the effects of the pandemic and launch economic recovery efforts.
Read IEG's Evaluation: State Your Business! An Evaluation of World Bank Group Support to the Reform of State-Owned Enterprises
Pictured above, clockwise from top left:
1. The Akuapem Rural Bank Ltd., founded in 1980, in the town of Mamfe, Ghana, June 19, 2006. Photo credit : Jonathan Ernst / World Bank
2. Kabul Afghanistan: Mirwais Zamkaniwal, 27 years old, Northwest Kabul Breshna Sub Station Manager, on site. Photo credit: Graham Crouch / World Bank
3. A female entrepreneur is visiting a bank in Vientiane. Vientiane, Lao PDR. Photo credit: Stanislas Fradelizi / World Bank
4. Interior of power plant. Kenya. Photo credit: Curt Carnemark / World Bank
Liberia: Integrated Public Financial Management Reform Project (PPAR)
PDF file
The project development objective of the Liberia Integrated Public Financial Management Reform Project (IPFMRP) was to improve the budget coverage, fiscal policy management, financial control, and oversight of government finances of the recipient. The project was restructured in 2016, but the project development objective remained unchanged. Four subobjectives are assessed for this review: (i) Show MoreThe project development objective of the Liberia Integrated Public Financial Management Reform Project (IPFMRP) was to improve the budget coverage, fiscal policy management, financial control, and oversight of government finances of the recipient. The project was restructured in 2016, but the project development objective remained unchanged. Four subobjectives are assessed for this review: (i) improve budget coverage, (ii) improve fiscal policy management, (iii) improve financial control, and (iv) improve oversight of government finances.
Ratings for the Integrated Public Financial Management Reform Project are as follows: Outcome was moderately unsatisfactory, Overall efficacy is modest, Risk to development outcome was substantial, Bank performance is moderately unsatisfactory, and Quality of monitoring and evaluation is negligible. Lessons from this project include: (i) Effective support for enhancing revenue mobilization and administration can benefit from combining technical assistance with logistical support. (ii) The use of PEFA composite indicators as results indicators is often not advisable. (iii) Superficial reviews and overoptimistic ratings in ISRs can negatively affect project implementation and outcomes. (iv) Effective and sustainable PFM reforms require continuous engagement to overcome political challenges.
What the World Bank Group’s Performance Results Cannot Tell Us About Development Outcomes
Web Resource
Many World Bank Group projects display strong performance. These results however tell us little about the Bank Group’s aggregate contributions and higher-level development outcomes—the results measurement systems are designed to tell a different type of performance story.
The World Bank Group’s (WBG) results measurement systems for projects, programs, and thematic areas are purposefully Show MoreMany World Bank Group projects display strong performance. These results however tell us little about the Bank Group’s aggregate contributions and higher-level development outcomes—the results measurement systems are designed to tell a different type of performance story.
The World Bank Group’s (WBG) results measurement systems for projects, programs, and thematic areas are purposefully designed.
Generally, they focus on collecting data on indicators that can be attributed to Bank Group interventions. They help verify whether targets have been met. They provide consistency in displaying results across portfolios. Importantly, they allow shareholders to hold the Bank Group accountable.
In our flagship report, IEG aggregates and analyzes much of this data to give the big picture on the Results and Performance of the World Bank Group (RAP 2020).
The 2020 RAP report finds that the World Bank and MIGA performs strongly overall, and that IFC’s decline in ratings that started ten years ago appears to have halted. We welcome you to dive deeper into how various arms and instruments of the Bank Group perform, and what factors help explain these trends.
At the same time, while being able to attribute results to Bank Group projects remains important for reasons of accountability, there is also a need to better capture how the institution contributes to lasting positive change in client countries.
We refer to this as higher-level development outcomes—social and economic change on the ground which Bank Group interventions cannot achieve singlehandedly, but which the institution can meaningfully add to, over time.
How well does the World Bank Group contribute to these sorts of development results? RAP 2020 finds that answering this question would require searching beyond the institution’s existing results measurement systems.
Limited evidence on higher-level outcomes is gathered
Following an analysis of the Bank Group’s results reporting—from projects to country programs to thematic priorities such as gender and climate change—the RAP 2020 arrived at the conclusion that the World Bank Group collects limited systematic evidence on its contribution to higher-level outcomes.
Higher-level outcomes stem from the interplay of different projects and types of Bank Group engagements—lending, knowledge, and convening—over time. The World Bank Group’s Board has requested more evidence on these sorts of development outcomes and how the interventions help achieve the Sustainable Development Goals (SDGs). Better evidence on higher level outcomes would also help with learning, reflections on strategy, and course corrections where needed.
Many of the World Bank Group’s existing corporate results measurement systems were designed to collect data needed for ratings and for process and compliance monitoring. These systems do what they were designed to do.
For example, individual projects’ self-evaluations mostly do a good job at capturing project-level results. IEG rates these self-evaluations, as part of a system that provides consistency in displaying results across sectors, across types of projects, and across time. Yet combining performance ratings from individual projects cannot give a clear picture of the important development outcomes to which the Bank Group contributes—the Bank Group’s contribution to higher-level outcomes amounts to more than the sum of its parts. And ratings are not the same as outcomes.
IFC, under its 3.0 strategy, has adopted tools to help it more clearly work toward outcomes. Through its AIMM system, IFC assesses all investments for their anticipated direct and indirect effects, including catalytic effects on markets. It is too early to tell how IFC’s new tools will influence outcome achievement and incentives. Although IFC can ensure alignment between projects and IFC’s higher-level goals because of its focused business model, the World Bank operates with objectives that are more diverse because of its diverse sector and country contexts.
Results in thematic areas focus more on compliance than outcomes
Consider gender and climate change. The Bank Group is committed to achieving important outcomes for its global work in key areas such as gender and climate change. Its measurement systems, however, place more emphasis on the input level—such as compliance with climate co-benefit commitments and gender tags—than on capturing changes arising from Bank Group operations.
The climate change results measurement system, for example, uses 35 targets and indicators to monitor how well the WBG integrates climate change into operations and strategies. 90 percent of these indicators relate to actions under the Bank Group’s control, including inputs (such as financing for climate action), internal processes (such as greenhouse gas accounting), and outputs (such as the number of activities that support cities with climate-related policies).
This approach is useful to track fulfillment of corporate commitments, drive accountability, and ensure operations adhere to process requirements. In a nutshell, the system creates incentives to mainstream climate action. But the system has limited focus on the quality of programs and on understanding the higher-level change the programs are meant to support.
What an outcome orientation would mean
Imagine a situation where staff spend less time checking boxes and collecting data that the Bank Group needs for its own internal reporting purposes and have more time engaging with clients on programs’ contribution to development outcomes. Imagine a situation where data, evidence, and systems oriented the Bank Group toward achieving higher-level outcomes. We call this outcome orientation.
An outcome orientation means both generating evidence on what works, what does not, and why; and using this feedback to engage clients and adapt programs to boost contribution to development outcomes.
Many parts of the World Bank Group are already studying their outcomes, learning from data and evidence, and adjusting interventions. Some of the corporate results systems could better support staff in this. IEG’s newly published evaluation on The World Bank Group Outcome Orientation at the Country Level finds that the country-level results measurement system does not effectively support teams in making course corrections to country programs.
***
The World Bank Group’s results measurement systems bring rigor and discipline to performance assessment, and there remains a strong case for assessing Bank Group achievements at the project level so that they can be accounted for in a focused way.
At the same time, more deliberately contributing to development outcomes requires shifting focus beyond the project level and beyond current results measurement and incentives structures. IEG is to this end committed to supporting the World Bank Group’s outcome orientation.
Read Results and Performance of the World Bank Group 2020
Public Utility Reform: What lessons can we learn from IEG evaluations in the energy and water sectors?
PDF file
This synthesis provides a review of operationally relevant findings and lessons from World Bank-supported utility reforms in the energy and water sectors, as identified in IEG evaluation products.This synthesis provides a review of operationally relevant findings and lessons from World Bank-supported utility reforms in the energy and water sectors, as identified in IEG evaluation products.
State Your Business!
Web Resource
This is IEG’s first systematic assessment of World Bank Group’s support for the reform of State-Owned Enterprises (SOEs), looking at what works and the factors of success. It parallels Bank Group efforts to provide more integrated support to SOE reform in client countries and to empower staff with new tools.
This is IEG’s first systematic assessment of World Bank Group’s support for the reform of State-Owned Enterprises (SOEs), looking at what works and the factors of success. It parallels Bank Group efforts to provide more integrated support to SOE reform in client countries and to empower staff with new tools.
Why evaluators should embrace the use of geospatial data during Covid-19 (Coronavirus) and beyond
Web Resource
Geospatial data encompass all information that is ‘geotagged’ to an exact geographical location on earth. This information can be remotely sensed from space—i.e. satellite imagery—but can also be collected from databases, surveys, project documents, and Monitoring & Evaluation (M&E) systems. The use of geospatial data on project variables has become an attractive solution to fill the void Show MoreGeospatial data encompass all information that is ‘geotagged’ to an exact geographical location on earth. This information can be remotely sensed from space—i.e. satellite imagery—but can also be collected from databases, surveys, project documents, and Monitoring & Evaluation (M&E) systems. The use of geospatial data on project variables has become an attractive solution to fill the void of field missions during the Covid-19 pandemic. Evaluators, however, were using geospatial data in evaluation even before travel was restricted. There is now an incredible opportunity for evaluators to use geospatial data more effectively and efficiently.
The last decade has seen rapid advances in all aspects of geospatial data, especially remote sensing data. First, satellite imagery has become more readily available, at lower (or zero) cost, and with higher quality. Terabytes of free and high-resolution raw data are created every single day. But more importantly, this raw satellite imagery is now rapidly processed into meaningful geospatial data by using machine learning. For example, raw images from the MODIS satellite are daily processed into geospatial data on land cover and forest fires that evaluators can use directly. Second, along with the revolution in big data, many data collection efforts—ranging from open-sourced platforms to household surveys—record the location of their observations. Similarly, more projects report on the geographical targeting of project investments. As a result, all of this geotagged information can be combined into one geospatial dataset. Finally, the analysis of geospatial data has become more efficient and user-friendly through open-source statistical programs.
Innovative geospatial data and software provide evaluators at the Independent Evaluation Group (and other evaluation functions) with unique tools to better address evaluation questions around the relevance and effectiveness of World Bank Group interventions. To assess the relevance of development interventions, evaluators can compare the spatial variation in a project variable with the spatial targeting of development interventions. For example, in a recent Country Program Evaluation for Mexico, IEG assessed whether investments to reduce poverty were directed towards areas with the highest poverty levels. Using regression analysis, and controlling for relevant exogenous variation, the analysis showed that World Bank support at the state level is positively correlated with the presence of the poorest 40% and is fairly independent of national public spending. To assess the effectiveness of development interventions, evaluators can use geospatial time-series to proxy changes in the project outcome indicators and construct a spatial counterfactual. A previous blog elaborated on how IEG exploits the spatial and temporal aspects of geospatial data in a robust impact assessment of World Bank projects in Mozambique, India, Ethiopia, and Madagascar.
Geospatial data also help overcome some of the methodological challenges to rigorously assess the sustainability of project impacts. Remote sensing data (e.g. satellite imagery) can provide unbiased and objective information on project outcomes on a granular level in every part of the globe. The availability of such data over time enables us to understand the evolution of particular variables over the entire life span of an intervention (and many years after the intervention). For example, IEG’s ongoing evaluation of Bank Group support to Municipal Solid Waste Management is using geospatial data to assess the sustainability of such support regarding the intended and unintended environmental and land use impact around supported landfill sites long after the respective projects have ended. Until recently, such analyses typically used to be beyond the scope (and feasibility) of a conventional (project) evaluation.
The use of geospatial data is, however, no silver bullet. Whether evaluations can apply geospatial data depends on the nature of the evaluand (e.g. the sector and type of intervention to be evaluated) as well as the analytical skills of the evaluators. Moreover, the objective and rigorous assessment of effectiveness using geospatial analysis is not sufficient on its own to assess why interventions are effective (or not). For example, it remains difficult to proxy political economy and human behavior aspects from available geospatial data. Ideally, any geospatial analysis requires some type of verification and triangulation ‘on the ground’. One of the most challenging constraints regarding the use of geospatial data in many multilateral and bilateral international development agencies (as well as some other organizations) is the disconnect between operations (which focuses on design and implementation) and evaluation. Independent evaluation functions are not directly involved in the intervention cycle (especially project design and implementation). Evaluators, therefore, have relatively little influence on the M&E frameworks of the interventions financed by their organization. A well-known consequence is that public and private investments often lack granular information on project implementation which complicates the use of geospatial data in evaluation afterwards.
Going forward, how can evaluation functions like IEG enhance their use of geospatial data? The first step is to focus on some of the low-hanging fruits. In the examples mentioned above, IEG has applied geospatial analysis to a particular set of interventions with a clear temporal and spatial nature. The analyses have been facilitated by the availability of numerous data portals with open access and ready-to-use geospatial data on a wide range of economic, environmental, and agroecological indicators. In some cases, evaluations have benefited from collaborative efforts with research colleagues. These examples have not only generated interesting and useful findings, they also provide useful lessons on the potential feasibility and desirability for conducting geospatial analysis in the framework of an evaluation. To better understand which geospatial data are useful and for which purposes, piloting new methods in the framework of different evaluation modalities should be encouraged. Investments in staff capacity development, hiring external experts, computing capacity and specialized software should be weighed against the results of these pilots.
The next step constitutes an organizational dialogue on the integration of geospatial data (collection and analysis) in the design and implementation of interventions. Evaluators can help make a stronger case for informed investments in geospatial data collection and analysis, leveraging the support from like-minded champions in research and operations departments. For example, the World Bank has launched two mobile applications (the Geo-Enabling Initiative for Monitoring and Supervision, GEMS, and the Smart Supervision App, SSA) that precisely register project locations and collect information that feeds into remote and real-time M&E systems. The GEMS initiative and the Geospatial Operations Support Team (GOST) also provide trainings and advice to build the capacity of clients and World Bank staff for remote project monitoring and supervision. This fits in the World Bank’s broader strategy to support client countries in developing the infrastructure, legal framework, and human capacity needed for the management and utilization of geospatial data. After the necessary ‘proof of concept’ experiences, a concerted organizational effort is needed to unleash the potential of geospatial data for better intervention design, implementation, and M&E.
Nigeria CLR Review FY14-19
PDF file
This review of the World Bank Group’s (WBG) Completion and Learning Review (CLR) covers the original period of the Nigeria Country Partnership Strategy (CPS), FY14-17, and the update and extension through FY19 as per the Second Performance and Learning Review (PLR) dated May 2018. The implementation of the CPS program was supported by 26 Bank operations with commitments of US$3.7 billion under Show MoreThis review of the World Bank Group’s (WBG) Completion and Learning Review (CLR) covers the original period of the Nigeria Country Partnership Strategy (CPS), FY14-17, and the update and extension through FY19 as per the Second Performance and Learning Review (PLR) dated May 2018. The implementation of the CPS program was supported by 26 Bank operations with commitments of US$3.7 billion under implementation at the beginning of the CPS and 38 new operations with commitments of US$9.4 billion. IFC invested in 28 projects for US$1.1 billion. MIGA issued three guarantees for US$549 million. The CPS design was well aligned with the challenges the country faced and the stated priorities of government. It also responded well to the challenges that arose during implementation. The CLR drew five lessons. Three of the lessons are: (i) achieving significant impact requires commitment beyond the horizon of a CPS, especially in areas such as energy and conflict mitigation; (ii) it can be difficult to accurately gauge the success or failure of results-based operations since they do not respond to traditional Bank tools for measuring success; and (iii) more care is needed in the selection of CPF objectives and results. In addition, IEG highlights the following two lessons from the CLR and builds on them: (i) The experience from expanding coverage of social assistance programs nationally under a common approach provides lessons that can be used to scale up engagements in other areas. Mainly, to combine the use of federal-level rules, policy coordination mechanisms, monitoring systems and data sharing with state-level program implementation and monitoring systems. (ii) Efforts to address design and implementation challenges included the creation of State Coordination Units to break logjams and the Multi-Sectoral Crisis Response Project (MCRP) to bring together efforts in infrastructure rehabilitation and service delivery in three conflictafflicted states. Further progress could entail absorbing and streamlining within the MCRP sectoral program delivery and institutional structures so as to reduce the number of PIUs and facilitate synergies.
The World Bank Group Outcome Orientation at the Country Level
Web Resource
This learning-focused evaluation provides a new vision of how to strengthen the World Bank Group's outcome orientation in countries.
This learning-focused evaluation provides a new vision of how to strengthen the World Bank Group's outcome orientation in countries.
Evaluation of International Development Interventions
Web Resource
This guide provides an overview of evaluation approaches and methods that have been used in the field of international development evaluation.
This guide provides an overview of evaluation approaches and methods that have been used in the field of international development evaluation.