Displaying 1 - 10 of 2924
Seriously funny: Using humor to spark deep conversations about evaluation
Web Resource
How humor helped IEG facilitate meaningful conversations about evaluation during a recent event.How humor helped IEG facilitate meaningful conversations about evaluation during a recent event.
World Bank Group Evaluation Principles
PDF file
This document, World Bank Group Evaluation Principles, sets out core principles for evaluation as well as underlying principles for selecting, conducting, and using evaluations, as relevant to the World Bank Group’s mission, and each institution’s mandate, system of governance, and operating environment. It is informed by international evaluation principles and good practice standards and builds Show MoreThis document, World Bank Group Evaluation Principles, sets out core principles for evaluation as well as underlying principles for selecting, conducting, and using evaluations, as relevant to the World Bank Group’s mission, and each institution’s mandate, system of governance, and operating environment. It is informed by international evaluation principles and good practice standards and builds on knowledge of current practices and processes around evaluation in the World Bank Group. In agreeing to this common set of principles, we aim to enhance development results by strengthening accountability and learning through evaluation.
Philippines: Social Welfare and Development Reform Project (PPAR)
PDF file
This is the Project Performance Assessment Report (PPAR) by the Independent Evaluation Group (IEG) of the World Bank Group on the Social Welfare and Development Reform Project (including additional financing) in the Philippines. The project had two objectives: (i) strengthen the effectiveness of the Department of Social Welfare and Development (DSWD) to efficiently implement the Pantawid Show MoreThis is the Project Performance Assessment Report (PPAR) by the Independent Evaluation Group (IEG) of the World Bank Group on the Social Welfare and Development Reform Project (including additional financing) in the Philippines. The project had two objectives: (i) strengthen the effectiveness of the Department of Social Welfare and Development (DSWD) to efficiently implement the Pantawid Pamilyang Pilipino Program (the CCT program, known as Pantawid); and (ii) strengthen the effectiveness of the DSWD to expand an efficient and functional National Household Targeting System of social protection programs.
Results for this Social Welfare and Development Reform Project are as follows: Outcome was highly satisfactory, Risk to development outcome was substantial, Bank performance was satisfactory, and Borrower performance was satisfactory. Lessons from the project include: (i) The success of a large, nationwide social protection program like Pantawid lies in creating and strengthening the operational and institutional systems needed to support it. (ii) Strong government ownership is critical to establishing and sustaining ambitious programs like Pantawid. (iii) The World Bank’s ability to bring global knowledge to bear and skillfully deploy a full technical engagement was key to success. (iv) Continuous monitoring and evaluation are essential to maintaining CCT programs like Pantawid and ensuring their constant evolution. (v) The quality of education and health, not just service utilization, is critical to achieve the expected gains in human capital. (vi) As for all CCTs, a graduation strategy is essential to ensure that the program delivers on longer-term benefits and acts as a stepping stone into more stable livelihoods.
Burundi CLR Review FY13-16
PDF file
This review of the World Bank Group’s (WBG) Completion and Learning Review (CLR) covers the period of the Country Assistance Strategy (CAS), FY13-16, and updated in the Performance and Learning Review (PLR) dated February 25, 2015. The World Bank Group’s (WBG) CAS had three focus areas: (i) improving competitiveness, (ii) improving resilience by consolidating social stability, and (iii) Show MoreThis review of the World Bank Group’s (WBG) Completion and Learning Review (CLR) covers the period of the Country Assistance Strategy (CAS), FY13-16, and updated in the Performance and Learning Review (PLR) dated February 25, 2015. The World Bank Group’s (WBG) CAS had three focus areas: (i) improving competitiveness, (ii) improving resilience by consolidating social stability, and (iii) strengthening governance. The CAS was broadly aligned with the Government’s Second National Poverty Reduction Strategy (PRSP II), 2012-2015, which seeks to improve governance, growth and job creation, social services, and environmental/spatial management. Specifically, the CAS focus areas and objectives supported PRSP II objectives on quality of economic infrastructure, promotion of the private sector and job creation, strengthening the social safety net, capacity building and improved
performance in the healthcare system, and fiscal management.
Ethiopia: Nutrition Project (PPAR)
PDF file
Although Ethiopia has achieved substantial progress in economic, social, and human development over the past decade, the ranking of its Human Development Index remains low. Malnutrition is widespread, and it lowers resistance to infections and affects the intellectual development of children and productivity among adults. The project development objectives were “to improve child and maternal care Show MoreAlthough Ethiopia has achieved substantial progress in economic, social, and human development over the past decade, the ranking of its Human Development Index remains low. Malnutrition is widespread, and it lowers resistance to infections and affects the intellectual development of children and productivity among adults. The project development objectives were “to improve child and maternal care behavior, and increase utilization of key micronutrients, in order to contribute to improving the nutritional status of vulnerable groups.” Direct beneficiaries consisted of pregnant and lactating women, and under-five children in food insecure regions with high malnutrition rates.
Ratings for the Nutrition Project are as follows: Outcome was satisfactory, Risk to development outcome was moderate, M&E Quality was substantial, Bank performance was moderately satisfactory, and Borrower performance was moderately satisfactory. Lessons from the project include: (i) The use of interactive approaches at the community level can facilitate behavior change. (ii) In very poor communities, CBN needs to complement behavior change interventions with income support to achieve the desired goals fully because behavior change also depends on the means to keep or to buy healthful and nutritionally rich food. (iii) Favorable institutional conditions, programmatic arrangements, and incentives facilitate the unfolding of multisectoral engagement. (iv) Integration of nutrition operations with an existing and institutionalized service delivery mechanism at the community level facilitates CBN implementation. (v) External collaboration with development partners, under government leadership, catalyzes international expertise and good practices that benefit and reinforce government policy and its nutrition agenda.
World Bank Group’s Support for Crisis Preparedness: Addressing Fiscal and Financial Sector Vulnerabilities (Approach Paper)
PDF file
The purpose of this evaluation is to assess the Bank Group’s support to client countries to enhance their preparedness for exogenous shocks through more systematic ex ante identification of vulnerabilities complemented by support to address these vulnerabilities. The evaluation is focused on the period between FY2010 and FY2018, after the global recession, to evaluate whether the Bank Group had Show MoreThe purpose of this evaluation is to assess the Bank Group’s support to client countries to enhance their preparedness for exogenous shocks through more systematic ex ante identification of vulnerabilities complemented by support to address these vulnerabilities. The evaluation is focused on the period between FY2010 and FY2018, after the global recession, to evaluate whether the Bank Group had integrated lessons from the global crisis. This evaluation aims to inform the design and use of WBG strategies, operations, diagnostics and knowledge products that support crisis preparedness in both low-income and middle-income countries.
Peru: Decentralized Subnational Roads Management
PDF file
This is a Project Performance Assessment Report (PPAR) on the Peru Regional Transport Infrastructure Decentralization Project, and the Decentralized Rural Transport Project. The objective of the Regional Project was to improve, through decentralization at the regional level the prioritization, the efficiency and effectiveness of regional transport interventions to contribute to Show MoreThis is a Project Performance Assessment Report (PPAR) on the Peru Regional Transport Infrastructure Decentralization Project, and the Decentralized Rural Transport Project. The objective of the Regional Project was to improve, through decentralization at the regional level the prioritization, the efficiency and effectiveness of regional transport interventions to contribute to regional development and poverty alleviation by enhancing transport conditions in the borrower’s territory. The Rural Project was to contribute to territorial development and the fight against rural poverty in the borrower’s territory by improving access of rural households and entrepreneurs to goods, social services, and income-generating opportunities through reduced transport costs and better rural transport infrastructure.
Ratings for the Regional Transport Infrastructure Decentralization Project are as follows: Outcome was moderately unsatisfactory, Risk to development outcome was significant, Bank performance was moderately unsatisfactory, and Borrower performance was moderately unsatisfactory. Ratings for the Decentralized Rural Transport Project are as follows: Outcome was satisfactory, Risk to development outcome was significant, and Bank performance was satisfactory, and Borrower performance was satisfactory. Lessons from these projects include: (i) Subnational governments need to own their road planning instruments to ensure their use. (ii) Ways to sustain the community-based microenterprises model for rural road maintenance need to be found. (iii) Road maintenance is essential all year round, and funding and bidding schedules need to be adjusted accordingly. (iv) Poverty impacts of rural roads projects are difficult to attribute. (v) f the road agency carries out activities that are outside its core responsibilities, it needs to involve the other ministries and government agencies that are responsible for these activities to ensure sustainability. (vi) Transferring successful solutions from one government level to another requires a careful contextual analysis and the subnational governments’ participation in decision making from the outset.
IEG Work Program and Budget (FY20) and Indicative Plan (FY21-22)
PDF file
To maximize its relevance and value added for the World Bank Group (WBG), IEG will align its work program with WBG strategic priorities. IEG also aims to maintain a clear line of sight with the WBG mission and the Sustainable Development Goals (SDGs), as well as with commitments made in the IBRD and IFC Capital Packages and in the context of IDA replenishments. Furthermore, IEG will keep an Show MoreTo maximize its relevance and value added for the World Bank Group (WBG), IEG will align its work program with WBG strategic priorities. IEG also aims to maintain a clear line of sight with the WBG mission and the Sustainable Development Goals (SDGs), as well as with commitments made in the IBRD and IFC Capital Packages and in the context of IDA replenishments. Furthermore, IEG will keep an increased focus on outcomes, countries, clients, and beneficiaries in its work, and aim to foster a greater outcome orientation throughout the WBG.
To achieve this strategic vision, IEG will focus its work program on the key development effectiveness questions that the institution and its clients are most concerned about. For each of these questions, we will strive to answer “why”, “how, “where”, “when”, and “for whom” specific interventions or programs have achieved results or not. By working more closely with operational units and other evaluation initiatives across the WBG, we will seek to significantly enhance IEG’s value added for the Board and WBG management.
The work program will be anchored around a series of “streams”, building evidence over time on connected themes and trying to bridge between project, country, sector and strategic impact: Fragility, Conflict and Violence (FCV), Gender, Maximizing Finance for Development, Human Capital, Climate Change, Growth and Transformation. In addition, IEG will work along an ‘effectiveness’ cross-cutting stream, aimed at examining systemic issues in WBG effectiveness, as well as working towards building a stronger outcome focus for WBG operations and strategies.
World Bank Group Support in Situations Involving Conflict-Induced Displacement
PDF file
This evaluation assesses the World Bank Group’s approach and support to countries hosting forcibly displaced populations—refugees and internally displaced persons (IDPs)—and provides evidence-based lessons to inform the Bank Group’s future role in this area. This evaluation assesses the World Bank Group’s approach and support to countries hosting forcibly displaced populations—refugees and internally displaced persons (IDPs)—and provides evidence-based lessons to inform the Bank Group’s future role in this area.
Five years ago the World Bank changed its operating model. Where do we go from here?
Web Resource
(Part 2 of a 2-part series about the findings of IEG’s evaluation Knowledge Flow and Collaboration Under the World Bank’s New Operating Model). Read Part 1.
Two important goals of
the World Bank reforms
Knowledge Flow is the process of bringing the right global knowledge to the right clients at the right time. Less advanced client countries want to know how more Show More(Part 2 of a 2-part series about the findings of IEG’s evaluation Knowledge Flow and Collaboration Under the World Bank’s New Operating Model). Read Part 1.
Two important goals of
the World Bank reforms
Knowledge Flow is the process of bringing the right global knowledge to the right clients at the right time. Less advanced client countries want to know how more advanced countries, such as Chile, Korea, Malaysia, or Singapore, handle technical reforms. This requires the free flow of people and knowledge across Regions and the Bank Group’s organizational boundaries. It also requires customizing knowledge to country contexts. And it requires strong knowledge production, curation, and management.
Integrated Solutions address complex issues with broad, cross-sectoral, and multiservice programs involving diverse tools and knowledge. Clients’ development challenges often cut across sectors and require diverse technical expertise from different sectors, disciplines, and both public and private sectors. To provide integrated solutions staff must collaborate across sectoral boundaries on multisector approaches, programs, or projects. Integrated solutions are meant to complement, not replace, traditional single-sector projects and programs.
In the previous post for this series, I explained how IEG found that the 2014 reforms have improved knowledge flows across regions, reduced silos, and deepened our expertise in specific areas, and that the Global Theme Groups have proven to be a useful addition to the operating model. These are all positive developments.
But as with any major change to a large organization’s structure, there have been some negative impacts. Some of the negative impacts can be traced to difficulties during the implementation of the reforms in 2014-15, as I will explain.
Challenges of Implementation
The new matrix organization has remained in flux because of managerial challenges. The central change management team was dispersed when the reform became effective in 2014. This caused discontinuity in reform implementation. There is no clear line of sight from reform plans laid out in 2013–14 and what has emerged today, after many subsequent adjustments. Having a central team to oversee reform implementation and use organizational performance data to inform course corrections could have benefited us.
The timing of expenditure savings also complicated reform implementation. Budget savings were not an original reform goal. Management introduced the “expenditure review” in 2014, unanticipated by most staff. The goal was sound - to increase IBRD’s financial capacity, bringing our expenditures in line with revenues. This review targeted budget cuts of $400 million in three phases over three years, implying also a need for staff reductions. The expenditure measures have been credited with paving the way for the 2018 capital increase. But their timing during organizational reforms resulted in challenges, such as staff morale suffering.
What are the problems with the current operational structure?
Although the objective of overcoming regional silos was sound, the new model has created other barriers to knowledge flow and collaboration.
Collaboration across boundaries is difficult. The GPs are competitive and have become new silos. Managers on both sides of the matrix incur high transaction costs when working across the GP structure.
The operating model works as intended when Country Directors and Program Leaders show leadership. Country Directors point to GP collaboration as a key pain point.
Implementing the crowded operational agenda is challenging. Our operations mainstream jobs, gender, climate change, maximizing finance, citizen engagement, fiduciary controls, and other issues. Mainstreaming requires collaboration across boundaries and tends to drive costs up. We task people with integrating, coordinating, and connecting across boundaries— Program Leaders, Global Leads, Global Themes Groups—but they often don’t have the authority or enough budget to do this.
There is a large gap between the reform’s aspirations to deepen knowledge and the current reality in many GPs. Some GPs have strategic approaches to knowledge; others do not. This is because of differences in the availability of trust funds and leadership support.
The mechanisms designed to pursue knowledge excellence have met with mixed results. A few GPs made their Global Solutions Groups work largely as intended, while others recast or disbanded the model. Global Leads have unclear roles and unfunded mandates.
There are concerns with quality assurance processes, specifically around contestability and the balance of oversight responsibility.
Why these weaknesses?
The reorganization did not change incentives, behaviors, and organizational culture.
Incentives for GP staff continue to favor own-managed lending, complicating collaboration. Managers without robust lending face fragmented and uncertain budget realities.
A simpler approachto the reorganizations may have been to retain existing groups and structures and realign incentives and reporting arrangements.
More could also have been done to create metrics for organizational effectiveness and use them to inform course corrections. Better data on the quality of services to clients could help us focus on results.
Where do we go from here?
Management is aware of the imperfections in the model, and, are using the findings of this evaluation along with other data to make course corrections.
As we move forward, it is important that we focus more on incentives, culture, and collaboration mechanisms than on structure.
Bank staff still have “change fatigue”—little appetite for another major reorganization. Will more changes to the structure, the org chart, improve things? The org chart can be redrawn with the stroke of a pen, as it has been many times since 2014, but does anyone think that will induce different behaviors?
To stimulate collaboration, we need more robust mechanisms to work across the matrix structure.
The two sides of the matrix need robust, authoritative connectors. As we explained in the evaluation, the existing pool of directors in the GPs is large enough to constitute a new cadre charged with connecting and arbitrating between GPs and Country Directors in each Region. Also, Program Leaders could be used more effectively to connect the two sides of the matrix. In our evaluation work, we saw many strong examples of Program Leaders helping to make cross-sectoral collaboration, integrated solutions, and complex client dialogue happen.
Since our evaluation was completed, senior management redefined the management roles in the Practice Groups, moving toward two senior roles: Global Directors and Regional Directors.
Any changes should maintain the global flow of staff and knowledge. The globally integrated nature of the new operating model should be preserved.
We also need to deepen our knowledge and continue to enhance knowledge flow. This calls for revamped incentives. Senior management could signal support for knowledge excellence. Metrics for knowledge uptake, impact, quality, and influence would help. There should be more contestability in quality assurance. More nimble budgeting arrangements and accelerated trust fund reforms would also contribute.
The main challenge going forward is to stimulate more collaboration under the new operating model while enhancing the initial gains on knowledge flow. By adjusting our incentives, culture, and collaboration mechanisms, we can achieve the level of integration and knowledge flow we will need to offer workable solutions to the increasingly complex problems the World Bank is tasked with helping to solve.
Read IEG's Evaluation: Knowledge Flow and Collaboration Under the World Bank’s New Operating Model