Search

Displaying 21 - 30 of 3057

Public Utility Reform: What lessons can we learn from IEG evaluations in the energy and water sectors?

PDF file
Public Utility Reform: What lessons can we learn from IEG evaluations in the energy and water sectors?
This synthesis provides a review of operationally relevant findings and lessons from World Bank-supported utility reforms in the energy and water sectors, as identified in IEG evaluation products.This synthesis provides a review of operationally relevant findings and lessons from World Bank-supported utility reforms in the energy and water sectors, as identified in IEG evaluation products.

State Your Business!

Web Resource
An Evaluation of World Bank Group Support to the Reform of State-Owned Enterprises, FY08-18
This is IEG’s first systematic assessment of World Bank Group’s support for the reform of State-Owned Enterprises (SOEs), looking at what works and the factors of success. It parallels Bank Group efforts to provide more integrated support to SOE reform in client countries and to empower staff with new tools. This is IEG’s first systematic assessment of World Bank Group’s support for the reform of State-Owned Enterprises (SOEs), looking at what works and the factors of success. It parallels Bank Group efforts to provide more integrated support to SOE reform in client countries and to empower staff with new tools.

Why evaluators should embrace the use of geospatial data during Covid-19 (Coronavirus) and beyond

Web Resource
Why evaluators should embrace the use of geospatial data during Covid-19 (Coronavirus) and beyond
Geospatial data encompass all information that is ‘geotagged’ to an exact geographical location on earth. This information can be remotely sensed from space—i.e. satellite imagery—but can also be collected from databases, surveys, project documents, and Monitoring & Evaluation (M&E) systems. The use of geospatial data on project variables has become an attractive solution to fill the void Show MoreGeospatial data encompass all information that is ‘geotagged’ to an exact geographical location on earth. This information can be remotely sensed from space—i.e. satellite imagery—but can also be collected from databases, surveys, project documents, and Monitoring & Evaluation (M&E) systems. The use of geospatial data on project variables has become an attractive solution to fill the void of field missions during the Covid-19 pandemic. Evaluators, however, were using geospatial data in evaluation even before travel was restricted. There is now an incredible opportunity for evaluators to use geospatial data more effectively and efficiently. The last decade has seen rapid advances in all aspects of geospatial data, especially remote sensing data. First, satellite imagery has become more readily available, at lower (or zero) cost, and with higher quality. Terabytes of free and high-resolution raw data are created every single day. But more importantly, this raw satellite imagery is now rapidly processed into meaningful geospatial data by using machine learning. For example, raw images from the MODIS satellite are daily processed into geospatial data on land cover and forest fires that evaluators can use directly. Second, along with the revolution in big data, many data collection efforts—ranging from open-sourced platforms to household surveys—record the location of their observations. Similarly, more projects report on the geographical targeting of project investments. As a result, all of this geotagged information can be combined into one geospatial dataset. Finally, the analysis of geospatial data has become more efficient and user-friendly through open-source statistical programs.  Innovative geospatial data and software provide evaluators at the Independent Evaluation Group (and other evaluation functions) with unique tools to better address evaluation questions around the relevance and effectiveness of World Bank Group interventions. To assess the relevance of development interventions, evaluators can compare the spatial variation in a project variable with the spatial targeting of development interventions. For example, in a recent Country Program Evaluation for Mexico, IEG assessed whether investments to reduce poverty were directed towards areas with the highest poverty levels. Using regression analysis, and controlling for relevant exogenous variation, the analysis showed that World Bank support at the state level is positively correlated with the presence of the poorest 40% and is fairly independent of national public spending. To assess the effectiveness of development interventions, evaluators can use geospatial time-series to proxy changes in the project outcome indicators and construct a spatial counterfactual. A previous blog elaborated on how IEG exploits the spatial and temporal aspects of geospatial data in a robust impact assessment of World Bank projects in Mozambique, India, Ethiopia, and Madagascar. Geospatial data also help overcome some of the methodological challenges to rigorously assess the sustainability of project impacts. Remote sensing data (e.g. satellite imagery) can provide unbiased and objective information on project outcomes on a granular level in every part of the globe. The availability of such data over time enables us to understand the evolution of particular variables over the entire life span of an intervention (and many years after the intervention). For example, IEG’s ongoing evaluation of Bank Group support to Municipal Solid Waste Management is using geospatial data to assess the sustainability of such support regarding the intended and unintended environmental and land use impact around supported landfill sites long after the respective projects have ended. Until recently, such analyses typically used to be beyond the scope (and feasibility) of a conventional (project) evaluation. The use of geospatial data is, however, no silver bullet. Whether evaluations can apply geospatial data depends on the nature of the evaluand (e.g. the sector and type of intervention to be evaluated) as well as the analytical skills of the evaluators. Moreover, the objective and rigorous assessment of effectiveness using geospatial analysis is not sufficient on its own to assess why interventions are effective (or not). For example, it remains difficult to proxy political economy and human behavior aspects from available geospatial data. Ideally, any geospatial analysis requires some type of verification and triangulation ‘on the ground’. One of the most challenging constraints regarding the use of geospatial data in many multilateral and bilateral international development agencies (as well as some other organizations) is the disconnect between operations (which focuses on design and implementation) and evaluation. Independent evaluation functions are not directly involved in the intervention cycle (especially project design and implementation). Evaluators, therefore, have relatively little influence on the M&E frameworks of the interventions financed by their organization. A well-known consequence is that public and private investments often lack granular information on project implementation which complicates the use of geospatial data in evaluation afterwards. Going forward, how can evaluation functions like IEG enhance their use of geospatial data? The first step is to focus on some of the low-hanging fruits. In the examples mentioned above, IEG has applied geospatial analysis to a particular set of interventions with a clear temporal and spatial nature. The analyses have been facilitated by the availability of numerous data portals with open access and ready-to-use geospatial data on a wide range of economic, environmental, and agroecological indicators. In some cases, evaluations have benefited from collaborative efforts with research colleagues. These examples have not only generated interesting and useful findings, they also provide useful lessons on the potential feasibility and desirability for conducting geospatial analysis in the framework of an evaluation. To better understand which geospatial data are useful and for which purposes, piloting new methods in the framework of different evaluation modalities should be encouraged. Investments in staff capacity development, hiring external experts, computing capacity and specialized software should be weighed against the results of these pilots. The next step constitutes an organizational dialogue on the integration of geospatial data (collection and analysis) in the design and implementation of interventions. Evaluators can help make a stronger case for informed investments in geospatial data collection and analysis, leveraging the support from like-minded champions in research and operations departments. For example, the World Bank has launched two mobile applications (the Geo-Enabling Initiative for Monitoring and Supervision, GEMS, and the Smart Supervision App, SSA) that precisely register project locations and collect information that feeds into remote and real-time M&E systems. The GEMS initiative and the Geospatial Operations Support Team (GOST) also provide trainings and advice to build the capacity of clients and World Bank staff for remote project monitoring and supervision. This fits in the World Bank’s broader strategy to support client countries in developing the infrastructure, legal framework, and human capacity needed for the management and utilization of geospatial data. After the necessary ‘proof of concept’ experiences, a concerted organizational effort is needed to unleash the potential of geospatial data for better intervention design, implementation, and M&E.

Nigeria CLR Review FY14-19

PDF file
This review of the World Bank Group’s (WBG) Completion and Learning Review (CLR) covers the original period of the Nigeria Country Partnership Strategy (CPS), FY14-17, and the update and extension through FY19 as per the Second Performance and Learning Review (PLR) dated May 2018. The implementation of the CPS program was supported by 26 Bank operations with commitments of US$3.7 billion under Show MoreThis review of the World Bank Group’s (WBG) Completion and Learning Review (CLR) covers the original period of the Nigeria Country Partnership Strategy (CPS), FY14-17, and the update and extension through FY19 as per the Second Performance and Learning Review (PLR) dated May 2018. The implementation of the CPS program was supported by 26 Bank operations with commitments of US$3.7 billion under implementation at the beginning of the CPS and 38 new operations with commitments of US$9.4 billion. IFC invested in 28 projects for US$1.1 billion. MIGA issued three guarantees for US$549 million. The CPS design was well aligned with the challenges the country faced and the stated priorities of government. It also responded well to the challenges that arose during implementation. The CLR drew five lessons. Three of the lessons are: (i) achieving significant impact requires commitment beyond the horizon of a CPS, especially in areas such as energy and conflict mitigation; (ii) it can be difficult to accurately gauge the success or failure of results-based operations since they do not respond to traditional Bank tools for measuring success; and (iii) more care is needed in the selection of CPF objectives and results. In addition, IEG highlights the following two lessons from the CLR and builds on them: (i) The experience from expanding coverage of social assistance programs nationally under a common approach provides lessons that can be used to scale up engagements in other areas. Mainly, to combine the use of federal-level rules, policy coordination mechanisms, monitoring systems and data sharing with state-level program implementation and monitoring systems. (ii) Efforts to address design and implementation challenges included the creation of State Coordination Units to break logjams and the Multi-Sectoral Crisis Response Project (MCRP) to bring together efforts in infrastructure rehabilitation and service delivery in three conflictafflicted states. Further progress could entail absorbing and streamlining within the MCRP sectoral program delivery and institutional structures so as to reduce the number of PIUs and facilitate synergies.

The World Bank Group Outcome Orientation at the Country Level

Web Resource
The World Bank Group Outcome Orientation at the Country Level
This learning-focused evaluation provides a new vision of how to strengthen the World Bank Group's outcome orientation in countries. This learning-focused evaluation provides a new vision of how to strengthen the World Bank Group's outcome orientation in countries.

Results and Performance of the World Bank Group 2020

Web Resource
Results and Performance of the World Bank Group 2020
This report, also known as RAP 2020, is an annual review of the development effectiveness of the World Bank Group. To provide new perspectives on performance, RAP 2020 also analyzes outcomes and discusses ways in which the Bank Group can continue to enhance its outcome orientation. This report, also known as RAP 2020, is an annual review of the development effectiveness of the World Bank Group. To provide new perspectives on performance, RAP 2020 also analyzes outcomes and discusses ways in which the Bank Group can continue to enhance its outcome orientation.

Evaluation of International Development Interventions

Web Resource
Evaluation of International Development Interventions
This guide provides an overview of evaluation approaches and methods that have been used in the field of international development evaluation. This guide provides an overview of evaluation approaches and methods that have been used in the field of international development evaluation.

Building on what is already there: Africa launch of the Global Evaluation Initiative

Web Resource
Participants of the Virtual Africa launch of the Global Evaluation Initiative
“The monitoring and evaluation process is essential for governance and the development of evidence-based policies,” announced Mozambique Vice Minster of Finance Dr. Carla Alexandra Oreste do Rosário Fernandes Louveir at the start of a virtual conference focused on the launch of the Global Evaluation Initiative and its potential role in Africa. “Monitoring and evaluation are powerful decision- Show More“The monitoring and evaluation process is essential for governance and the development of evidence-based policies,” announced Mozambique Vice Minster of Finance Dr. Carla Alexandra Oreste do Rosário Fernandes Louveir at the start of a virtual conference focused on the launch of the Global Evaluation Initiative and its potential role in Africa. “Monitoring and evaluation are powerful decision-making tools for managers,” added the Vice Minister. The Global Evaluation Initiative (GEI) brings together a broad and inclusive coalition of governments, citizens and experts to close a global gap in monitoring and evaluation capacities. According to a recent report, more than half of all countries now have high-quality national development strategies and almost all of them are aligned with the Sustainable Development Goals (SDGs), but only one third of them have data and systems to track implementation of their policies. Responding to an urgent need A sense of urgency was palpable during the launch event, “Evidence and the Road to 2030,” which brought together government and independent evaluation experts from across Africa, as well as representatives of international organizations and donor countries. The Sustainable Development Goals were already off track as the 10-year countdown to the 2030 Agenda began this year, and the COVID-19 pandemic is derailing them further, threatening to push millions into poverty. “We need reliable evidence to guide our path to economic recovery, “ said event participant Godfrey Mashamba, South Africa’s Deputy Director General of the Department of Planning and Monitoring and Evaluation. Monitoring and evaluation (M&E) can deliver invaluable evidence on what is working and what is not. They can be a powerful compass, helping governments learn from experience and use the lessons to adjust course, scale or target public policies more effectively. There is strong demand from countries in Africa and around the world to strengthen their M&E capacities as critical for rebuilding better from the pandemic and getting back on track to the SDGs. Linking national, regional and global knowledge GEI aims to respond to the demand by building on the capacities, experiences and knowledge of local actors and matching these with coordinated support and global knowledge, according to representatives of two organizations at the heart of the  partnership who spoke during the event: Alison Evans, Director-General of Evaluation at the World Bank Group, and Oscar A. Garcia, Director of the Independent Evaluation Office of the United Nations Development Programme (UNDP). For example, Cabo Verde, a small country of 10 islands off Western Africa, has many sources of data but does not yet have the institutional capacity to process and take advantage of this information, according to panelist Gilson Pina, National Director of Planning at the Cabo Verde Ministry of Finance. GEI will help train and build the capacities of staff to use the data of a new M&E platform that his planning department is building, Pina said. “The initiative’s goals coincide with our own goals,” Pina said during his intervention at the conference. Beyond building basic capacity, GEI should serve to “support a culture of evaluation” and encourage shared progress for African countries, said panelist Abdoulaye Gounou, Head of Benin’s Office for the Evaluation of Public Policies and Analysis of Government Action. “The partnership on evaluation is a very powerful tool for institutionalizing and promoting evaluation in our countries,” Gounou said. “We are learning through our peers and we are progressing together.” The partnership on evaluation is a very powerful tool for institutionalizing and promoting evaluation in our countries. We are learning through our peers and we are progressing together. Abdoulaye Gounou, Head of Benin’s Office for the Evaluation of Public Policies and Analysis of Government Action. Independent evaluators in Africa can also play a role in expanding M&E capacities in the region and developing African approaches to evaluation, said panelist Fazeela Hoosen, co-chair of the Young and Emerging Evaluators Network of the African Evaluation Association. As the next generation of evaluators, “Young and emerging evaluators are the ground or fertile soil to embed the seeds of what we have been talking about,” she said. Watch a re-play of the Africa Launch of the Global Evaluation Initiative {"preview_thumbnail":"/sites/default/files/Data/styles/video_embed_wysiwyg_preview/public/video_thumbnails/lA9WZ9aWxAg.jpg?itok=bALL_eQR","video_url":"https://youtu.be/lA9WZ9aWxAg?t=3734","settings":{"responsive":0,"width":"854","height":"480","autoplay":0},"settings_summary":["Embedded Video (854x480)."]}   Regional and global cooperation GEI aims to collaborate with the diverse set of stakeholders involved in the development of M&E capacity, from government officials to independent evaluators, and to serve as a global platform for the curating and sharing of local and global M&E knowledge. The partnership has already been in touch with the African Evaluation Association (AfrEA), an umbrella organization for individual evaluators in countries that lack national evaluation associations, said panelist Rosetti Nabbumba of Uganda, president of AfrEA’s Board of Directors. GEI has begun talking with AfrEA about working together to compile a database of African evaluators. “I’m looking forward to a very collaborative arrangement,” with GEI, Nabbumba said. The Swedish International Development Agency (Sida) also has high expectations for GEI, with the hope that it will take the evaluation capacity agenda forward with a focus on country-led efforts according to panelist Pernilla Rafiqui, Sida’s Senior Program Manager for Capacity Development. Rafiqui described GEI as “both timely and well conceptualized,” and added, “we expect GEI to contribute toward better policies for better lives for poor and vulnerable people, and we very much look forward to making this happen.” In closing remarks, Dr. Anthony Akoto Osei, Ghana’s Minister for Monitoring and Evaluation, emphasized the urgent need in the uncertain context of COVID-19 for robust M&E systems that enhance the chances of success of government policies. “The need for and use of evidence in support of decision-making have never been as important as it is now,” Dr. Osei said. “The required improvements in M&E need to be underpinned by what is already in place. The GEI is an example of that kind of strategy, in that it seeks to build on what is already there, and to operate collaboratively and through partnerships.” Learn more about GEI Watch the re-play of the launch event in English, French, or Portuguese.

The deepening and broadening discussion on assessing ‘Who Benefits?’ in evaluation practice

Web Resource
High peak demand for energy in FYR Macedonia can result in power outages around the country. Here a young child does homework by lamplight in a home outside of Skopje. Photo: Tomislav Georgiev / World Bank
After attending the UNDP National Evaluation Capacities and the American Evaluation Association Conferences in 2019, I wrote a blog Are Evaluators Ready to Answer Question: Who Benefits? The answer was that increasingly evaluation practice was grappling with ‘Who Benefits’ through increasingly engaging with issues such as inequality, inequity, the differing impacts of climate change, and the Show MoreAfter attending the UNDP National Evaluation Capacities and the American Evaluation Association Conferences in 2019, I wrote a blog Are Evaluators Ready to Answer Question: Who Benefits? The answer was that increasingly evaluation practice was grappling with ‘Who Benefits’ through increasingly engaging with issues such as inequality, inequity, the differing impacts of climate change, and the role of youth. This year I attended the virtual American Evaluation Association (AEA) conference interested in how the debates on assessing benefits had developed. The theme of the AEA conference was ‘How Will You Shine Your Light’, which relates to the idea that evaluators shine their light for the purpose of “improving conditions for others to shine theirs”. Throughout, the conference focused on equity, the fair distribution of benefits. The presidential track of the conference reflected on equity through discussions on restorative practice, racism in qualitative practice, disability in evaluation, community-based foresight practices, climate change and presenting a framework for assessing community harmony. Outside of the presidential strand, presentations also focused on gender equity, equity-based policy advocacy and equity and inclusion. The emergence of equity as a major theme in evaluation practice reflects trends where evaluation gatherings embrace a diversity of perspectives, including indigenous, youth and social justice perspectives. Many of the presentations at the conference reflected critically on approaches to engage marginalized groups and called for more context specific and culturally appropriate methods. Some of the interesting resources that I would reference again as I develop evaluations were related to youth, the rubric developed to measure love, equity in data science and foresight techniques. Doing No Harm in Evaluation Harm is not a word that evaluators have engaged with deeply very often. The presenters of this Think Tank initially came together to reflect on harm in evaluation practice when organizations that were meant to protect rights but caused harm were exposed. Over a period of two years we have further understood that evaluation can cause harm through its: handling of allegations of sexual exploitation; inaccurate findings; privileging of voices; myopically focusing on donor values; careless storing of data; cultural insensitivity; and not evaluating harm. This think tank introduces these issues and seeks to identify where our principles and standards need updating to better reflect the potential for harm. The Think Tank will engage participants in a participatory reflection process aimed to develop adaptations for where our principles are found wanting. My understanding of the ‘Who Benefits’ question was especially deepened in facilitating a Think Tank entitled Doing No Harm in Evaluation that drew on a two-year action learning process. Discussing harm is the ‘shadow’ of the equity and benefits discussions, as one of our discussants highlighted. Evaluators and project managers who have the best of intentions and seek to diversify who, what and how benefits accrue can do harm – think of abuse in humanitarian settings. In debating harm, the participants went beyond concerns with ethics panels and informed consent. They questioned how harm could be caused by funding and commissioning mechanisms, a tendency to drop into contexts with limited preparation, and the answering of questions that were defined far from intended beneficiaries. Emerging research that was presented within the Think Tank made the argument that in evaluation there is inherent reductionism and carcerality, which requires changing listening practices, reciprocity with knowledge, and undertaking truth-telling on harm. The online chat during the think tank produced just less than 300 comments in 45 minutes. In Interpreting these afterward the following five themes emerged: Trade-Offs – In identifying who benefits, harm is embedded. Consequently, trade-offs between benefit and harm should be acknowledged in the evaluation processes, for example, in the (i) methods deployed; (ii) scope of the evaluation; (iii) engagement with commissioners; (iv)  management process. Reflexivity – Evaluators hold a position of privilege in defining benefit and should give attention and attend to personal growth in their own judgements, beliefs and practices as benefits can be defined in diverse ways. Trauma informed and restorative practices – Identifying who benefits may also bring an evaluation into contact with past traumas. Trauma-informed and restorative practices arose as approaches to help account and tell the truth on the harms of past practice while providing a framework to help manage trade-offs in current evaluation practice. Inclusivity – To understand who benefits requires improvements in h who evaluation practice includes, through listening skills and incorporating feedback loops. Creating space for listening will require responding to new and unexpected information on diverse perspectives. Transformative lens – Working with a transformative evaluation perspective helps to identify different benefits and harms by reinforcing attention to voice and liberation in discussions. Transformation provides potentially a useful ethical principle in reframing practice to better respond to harm. These themes and the discussions presented here reflect a small part of exchanges on equity that evaluators engaged in during the conference. Terms such as trade-offs, reciprocity, restorative, reflexivity, transformative are increasingly eminent in evaluation practice and represent a deepening discussion amongst evaluators to engage with ‘Who Benefits?’   Pictured above: High peak demand for energy in FYR Macedonia can result in power outages around the country. Here a young child does homework by lamplight in a home outside of Skopje. Photo: Tomislav Georgiev / World Bank

Jamaica: Hurricane Dean Emergency Recovery Loan (PPAR)

PDF file
Jamaica is highly exposed to natural disasters. The negative impacts on economic development and social well-being are exacerbated as approximately 82 percent of Jamaica’s population lives within 5 kilometers of the coast, increasing the relative vulnerability of residents, major infrastructure, and the housing stock. Hurricane Dean made landfall in Jamaica on August 19, 2007, causing economic Show MoreJamaica is highly exposed to natural disasters. The negative impacts on economic development and social well-being are exacerbated as approximately 82 percent of Jamaica’s population lives within 5 kilometers of the coast, increasing the relative vulnerability of residents, major infrastructure, and the housing stock. Hurricane Dean made landfall in Jamaica on August 19, 2007, causing economic losses of roughly $329 million. The hurricane resulted in significant and extensive damage to primary and early childhood schools, community-based health clinics, and parochial and agricultural feeder roads in directly impacted parishes. In the aftermath of the hurricane, Jamaica’s Ministry of Finance confirmed that the recovery would require financial support from multiple sources, both national and international. In that context, the government of Jamaica approached the World Bank to support reconstruction works in poor communities affected by Hurricane Dean. The general aim was the reestablishment of prehurricane living conditions in these communities through the implementation of specific local infrastructure projects that would directly improve the conditions of the most vulnerable populations. Given the ongoing emergency, the World Bank and the government of Jamaica agreed to sign an emergency recovery loan to expedite the disbursement of resources. Additionally, the World Bank and the government of Jamaica agreed that the Jamaica Social Investment Fund (JSIF) would be the implementing agency. Ratings for the Hurricane Dean Emergency Recovery Loan are as follows: Outcome was moderately satisfactory, Risk to development outcome was moderate, Bank performance was moderately satisfactory, and Borrower performance was satisfactory. Lessons from this project include: (i) Using existing agencies with a proven track record can be an effective approach for implementing emergency response projects. (ii) When designing rehabilitation works, close consultation with users can ensure the provision of better services. (iii) Expectations need to be managed as there are limits to how much progress can be made on disaster risk reduction or emergency preparedness under an emergency operation.