Complementary Interventions for Agrifood System Development – Insights and Lessons
Web Resource
Agrifood systems are key towards achieving the Sustainable Development Goals, including ending hunger and malnutrition, poverty, and addressing climate change. Yet, despite their importance, agrifood systems face multiple challenges, including low productivity, inadequate market access for small farmers and producers particularly in low-income countries (LICs) and vulnerability to climate change Show MoreAgrifood systems are key towards achieving the Sustainable Development Goals, including ending hunger and malnutrition, poverty, and addressing climate change. Yet, despite their importance, agrifood systems face multiple challenges, including low productivity, inadequate market access for small farmers and producers particularly in low-income countries (LICs) and vulnerability to climate change.
The World Bank Group aims to address these challenges through holistic interventions that increase the productivity, inclusion, and sustainability of agrifood systems. A recent evaluation by the Independent Evaluation Group (IEG) of the World Bank Group’s support for agrifood system development over ten years (2010-20) found that when market access is constrained, complementary interventions that pair support for enhancing sustainable production on the supply-side with that to improve market access on the demand-side can increase productivity and the overall effectiveness of interventions.
Low productivity remains a major challenge for small farmers, especially in sub-Saharan Africa. For example, staple crop productivity in sub-Saharan Africa and LICs is about one-third of the level in upper-middle-income countries. Smallholders and small producers, especially in LICs find it challenging to increase productivity because of their limited access to markets, including agri-finance to invest in modern inputs and technologies. Low yields and weak market integration lead to fragmentation of production, low incomes, food insecurity, and pervasive poverty. When markets are underdeveloped and poorly integrated, low productivity also leads to higher food prices which, in turn, leads to a high cost of living for the rural and urban poor. Underdeveloped markets perpetuate subsistence production and undermine the ability of producers to respond to market signals or diversify production into higher value products with growing demand – limiting opportunities for inclusive growth in the rural sector.
Complementary interventions that helped increase productivity included access to finance to buy better inputs, such as fertilizers and feed, invest in small-scale irrigation and farm equipment, or in new technologies, such as improved seeds or crossbred cows. It also included improved access to services (e.g., extension in the form of knowledge, research, or technology support such as artificial insemination or provision of market information), and support to improve access to and participation in markets and value chains. Investing in sustainable irrigation and productive climate-smart practices helped amplify these benefits to producers and increased productivity through higher yields and multiple harvests during the year. The evidence across countries showed that improving complementarity of supply and demand-side interventions was key to maximizing benefits from Bank Group interventions.
Two examples from the evaluation illustrate the significance of complementary interventions. In the first example, the Ethiopia Agricultural Growth Project (AGP I) supported farmers using a complementary approach. The project helped them access crop and livestock technologies, and climate-smart practices, including small-scale irrigation. It also helped them access markets for their produce. The project supported improved irrigation on over 10,000 hectares of farmland and increased the marketed surplus of crops that benefited more than 58,000 farmers, including over 12,000 women and 6,000 young people. In the second example, the Malawi Irrigation, Rural Livelihoods, and Agricultural Development Project focused on supply- side interventions and had a limited impact on productivity. Smallholder farmers struggled to find sustainable market outlets for their maize and rice produce, leading to greater volatility in producer prices. As a result, while productivity of both maize and rice improved initially, it stagnated or became more volatile over time.
Supply side interventions may succeed when market access is not constrained. For example, the Integrated Agricultural Productivity Project in Bangladesh helped increase productivity by supporting technology development and adaptation, including improved practices for conserving water and making irrigation more efficient. The project helped build the capacity of farmers and provided extension support, which led to adoption of better varieties of crops, livestock, and fish breeds. The improved technologies benefited about 51,000 farmers. Milk productivity more than doubled, milk consumption increased by 96 percent, milk sales increased fourfold, and milk sales earnings increased fivefold. The interventions significantly increased the seasonal earnings of beneficiary farmers from sale of crops, thereby, enhancing inclusion. The project also improved sustainability by putting over 27,000 hectares under better irrigation practices.
Producer organizations play a key role in improving the access of smallholder farmers to markets and services by connecting them with other input providers and more organized buyers. Producer organizations come in several forms, such as common interest groups and cooperatives in Kenya and Ethiopia, farmer groups in productive alliances in Peru and Bolivia, and dairy and livestock cooperatives in India and Vietnam. IEG also found that most projects that aimed to increase inclusion also supported such producer organizations to help farmers access inputs, technologies, services, and markets. Strengthening the capacity of producer groups for better targeting and inclusion also increased the participation of women and youth.
Making the transition from informal groups into producer cooperatives or enterprises can, however, be more challenging for producers, who are engaged in low-value commodities. In Kenya, only about one-third of the newly established cooperatives supported by the Agricultural Productivity and Agribusiness Project were active after the project closed, except for the few involved in high value products such as dairy. Similarly, in Ethiopia the newly established cooperatives were more successful when they were able to connect with processors and were able to benefit from value chains.
When farmer groups face multiple challenges to succeed in accessing markets, timely support to help them transition to producer cooperatives and enterprises can be effective.
IEG found that the Ethiopia Agricultural Growth Project helped connect three producer groups – Guguma Buraro, Guguma Buko, and Guguma Wube – in Meliga woreda in Oromia region - with the Assela Malt Factory. This provided the groups an incentive to invest in inputs and increase their productivity. Their supply of malt barely increased by over ten-fold from 62 tons in 2015 to 730 tons in 2017. This allowed the groups to become formal producer cooperatives and expand their access to finance and services. Targeting bottlenecks on both the supply (production) and demand (market) sides of the malt barley value chain, facilitated the transition from informal groups into market-oriented producer cooperatives.
The above lessons can help the World Bank Group better support its clients in addressing the challenges of low yields, weak market access and inadequate integration of smallholder production into markets and value chains which often leads to low productivity. Providing complementary support is vital in transitioning toward more productive, inclusive, and sustainable agrifood systems and can contribute to finding viable solutions to the current global food crisis.
See also: Building inclusive, productive, and sustainable agrifood systems
The World Bank Group in Madagascar
Web Resource
This evaluation assesses the development effectiveness of the World Bank Group’s engagement with Madagascar during Fiscal Years 2007–21.
This evaluation assesses the development effectiveness of the World Bank Group’s engagement with Madagascar during Fiscal Years 2007–21.
How the Wapenhans report came to change the World Bank 30 years ago
Web Resource
Thirty years ago, the Wapenhans report sounded the alarm over the World Bank’s worsening portfolio quality and sparked changes in how the Bank manages the quality and results of its projects. This two-part blog series takes stock of where the World Bank is 30 years after this influential evaluation.
The 1992 Wapenhans report – formally known as “Effective Implementation: Key to Development Show MoreThirty years ago, the Wapenhans report sounded the alarm over the World Bank’s worsening portfolio quality and sparked changes in how the Bank manages the quality and results of its projects. This two-part blog series takes stock of where the World Bank is 30 years after this influential evaluation.
The 1992 Wapenhans report – formally known as “Effective Implementation: Key to Development Impact” – was arguably the most influential evaluation ever at the World Bank. Many of the institution’s core quality assurance instruments and processes were created in response to the report’s findings. Although many newer staff today have not heard of it, the report was part of the induction of new Bank staff for many years.
The context of Wapenhans
The report was written by a team formed for just this purpose called the Portfolio Management Task Force, led by Willi A. Wapenhans, Vice President of the World Bank. The Operations Evaluation Department (OED) – precursor to the Independent Evaluation Group (IEG) – contributed but did not get to write the report.
The report’s inside history, documented in the archives, makes for fascinating reading 30 years later as it captures how the report came to be so influential.
Lewis Preston, who was new as President of the World Bank wanted an unvarnished view of the quality of the portfolio of projects under implementation. The Bank did not have a good measure of its development impact at the time. In the 1970s, World Bank President Robert McNamara had led the transformation of the Bank into a development institution; he created the Operations Evaluation function to begin assessing the institution’s development effectiveness, but the Bank did not yet have a full view of how well its projects led to sustainable development in client countries.
Wapenhans identifies a steady decline in portfolio performance
The share of projects with major problems nearly doubled from 11% in Fiscal Year (FY)81 to 20% in FY91. Some 30% of projects in their fourth and fifth year of implementation were reported as having major problems.
The Wapenhans report placed much of the responsibility for weak portfolio quality on overly optimistic project approval by the Bank, driven by an “approval culture” and inadequate attention to risk and implementation planning. Wapenhans also identified a bias for complex projects with many components and co-financiers as contributing to weak quality at entry.
Further, Wapenhans advised that the Bank figure out how it should support project implementation. The division of labor with borrowers was unclear at the time, and there were major issues with procurement. Country assistance strategies did not factor in portfolio performance. Bank projects did not systematically address country and sector specific obstacles to implementation.
Lastly, the report concluded that the Bank’s evaluation system did too little to assess project outcomes and their sustainability once projects had closed, weakening the Bank’s ability to learn about what works and to ensure accountability for outcomes.
The findings resonate across the institution
Although some of the responses from Bank management were defensive, the extensive consultations that Wapenhans had caried out across the Bank and with its borrowers had created a strong head of steam for the report’s findings. Many operational staff and managers agreed that there were deep problems in project quality and implementation.
Reading through the responses to the report, there is a palpable sense that the process of producing and consulting on the report opened the floodgates for staff to share concerns that had been known for a long time but not openly acknowledged. Wapenhans used OED project ratings and other data on operational quality to make a compelling case that the portfolio was not in good shape.
The takeaways are still relevant for the World Bank Group today. The Wapenhans report was a watershed moment in how the Bank manages its portfolio. In response to the report, the Bank and OED created metrics of quality at entry, quality of supervision, and quality of M&E that are still in use today. It created a Board Committee on Development Effectiveness (CODE), Development Effectiveness units in the Regions, and eventually also the Quality Assurance Group, since disbanded. The Bank also strengthened its country assistance strategy process and linked it to assessment of portfolio quality.
Evaluation as a catalyst for change
All organizations face internal challenges from time to time. An organization like the World Bank with its smart, reflective, mission-driven people has the capacity to identify and correct organizational challenges from within. All the while, improvement processes need a catalyst. In this case, the catalyst was Wapenhans’ comprehensive evaluation backed up by Preston’s firm support.
Today, it is the role of IEG to produce evaluations of the World Bank Group’s development effectiveness. We see a reflection of our own experience in the story of the Wapenhans report: that evaluation based on solid data often helps organizations, by providing a firmer diagnostic underpinning to internal discussions about results and effectiveness that would otherwise be more anecdotal. Evaluations provide a solid ground of evidence to support organizational change and strategy development.
In conclusion, the Wapenhans report led to sustained changes in how the World Bank Group manages for quality and results. This was possible because Wapenhans got many things right: the report was timely, full of credible data and solid analysis, informed by extensive engagement, and followed up by changes in management practices. These have all become essential elements of evaluation good practices that IEG adheres to.
First photo: Interview with Mr. Willi A. Wapenhans, left, and Mr. P. Karieti, right, in 1983. Photo credit: World Bank.
🎧 Lessons Unwrapped from Tackling Plastic Waste
Web Resource
Every year, the world generates 2 billion tons of trash, including 400 million tons of plastic. Most of this waste is mismanaged, piling up and flowing into our oceans, adding to greenhouse gas emissions and land and water pollution. A long-term solution requires the world to shift to a circular economy. What does circularity entail? What can we learn from global efforts to tackle solid waste, Show More
Every year, the world generates 2 billion tons of trash, including 400 million tons of plastic. Most of this waste is mismanaged, piling up and flowing into our oceans, adding to greenhouse gas emissions and land and water pollution. A long-term solution requires the world to shift to a circular economy. What does circularity entail? What can we learn from global efforts to tackle solid waste, and in particular plastics?
Host Jeff Chelsky talks trash with Steve Fletcher, Professor of Ocean Policy and Economy at the University of Portsmouth, to take stock of lessons from a waste crisis that is disproportionately affecting people in poverty.
Listen on Spotify, Apple Podcasts, or Stitcher.
Related Resources
IEG Evaluation: Transitioning to a Circular Economy: An Evaluation of the World Bank Group's Support for Municipal Solid Waste Management (2010-20)
Blog: Towards a circular economy: Addressing the waste management threat
Addressing rural water supply gaps with performance-based financing: lessons from Vietnam
Web Resource
The share of the world’s rural population with access to safely managed drinking water services reached 60% by 2020, but still lags behind a comparative rate of 85% among urban populations, according to the World Health Organization and the United Nations International Children’s Emergency Fund’s Joint Monitoring Programme. Bridging this gap by increasing rural access to improved and sustainable Show MoreThe share of the world’s rural population with access to safely managed drinking water services reached 60% by 2020, but still lags behind a comparative rate of 85% among urban populations, according to the World Health Organization and the United Nations International Children’s Emergency Fund’s Joint Monitoring Programme. Bridging this gap by increasing rural access to improved and sustainable water supply and sanitation services is critical to developing human capital to unleash the growth potential of rural areas.
The World Bank’s synthesis report on the policy, institutions, and regulation (PIR) approach in the water and sanitation sector noted that financial incentives can be enhanced through performance-based financing (PBF) mechanisms and instruments, such as the World Bank’s Program for Results (PforR). Incentives can be enhanced through the use of performance-based contracts with the private sector, which involves payment to contractors being directly linked to the delivery of timely and quality results. This model was recently used in Vietnam, as part of the first operation for the country and the World Bank’s Water Global Practice to adopt the PforR financial instrument to improve rural access to water and sanitation services. An evaluation of the results and performance of the operation by the Independent Evaluation Group (IEG) identifies a number of lessons to guide the future use of performance-based financing in both the water and sanitation, and other sectors.
Vietnam has had an impressive record of inclusive economic growth since 1980s, but poverty has remained concentrated in rural areas and among ethnic minorities. Although rural access to water and sanitation improved along with economic growth, challenges remained regarding the sustainability of the initiatives that provide water services and hygiene. To address those sectoral challenges, in 2011 the government of Vietnam updated the National Target Programs (NTPs) for Rural Water Supply and Sanitation to strengthen community participation, demand-responsive approaches, and cost recovery. The World Bank has been supporting Vietnam’s rural water supply and sanitation (RWSS) sector for more than 20 years and launched the Results-Based Rural Water Supply and Sanitation under the National Target Program (RWSS PforR), introducing a results-based approach in fund allocations and disbursements.
What Worked, and Why
The PforR was a good instrument for addressing the dominance of and collusion among state-owned enterprises in contract procurement and management. The RWSS PforR promoted competitive bidding to enhance transparency and accountability in public agencies’ procurement processes. The results-based financing approach addressed the frequent delays in water scheme constructions. The PforR promoted results-based financing and independent results verification, which led to: (i) transferring risk associated with construction delays to service providers and contractors to motivate them to deliver results quickly and cost-effectively; (ii) incentivizing implementing agencies to select target areas based on criteria including the households’ willingness to pay for water connections; and (iii) making the contractors accountable for ensuring quality, transparency, and accountability throughout the construction process.
The PforR also contributed to diversifying financing for rural water service delivery by promoting private sector–led service delivery models. Before the RWSS PforR, the rural water systems were seen as scattered and difficult to manage with a low potential for profit, and consequently private investment in most of the eight participating provinces was also low. The World Bank used the PforR to try to encourage the provincial governments to acknowledge that financing from, and relationships with, the private sector were fundamental to provincial investments. The involvement of private enterprises contributed to increased revenues and higher staff productivity, which led to improved cost recovery ratios for water supply schemes The privately managed schemes in rural areas of Vietnam had higher profits that averaged 35% of overall revenues, compared with 20% or less for other types of service providers. This was in line with the overall trend of an increase in private sector investments in the RWSS sector. Public-private partnerships in water and sanitation increased from $88.9 million in 2016 to $159.14 million in 2019. Moreover, privately managed water schemes had higher staff productivity. On average, one worker in a private enterprise distributed 3,850 cubic meters of water supply monthly, compared with 970 cubic meters for staff in community- or public-led service provision.
What Didn’t Work, and Why
In results-based operations, the drive to achieve the disbursement-linked indicator (DLI) targets could result in prioritizing communities that are relatively more advanced and already have lower poverty rates at program start-up. Provincial governments prioritized communes already on the way to meeting targets, rather than those with the greatest need. To ensure receipt of fund disbursements, the operations may focus on producing tangible results as early as possible, thus incentivizing the assignment of a lower priority to more complex activities. However, the latter, more complex activities may be critical to the broader development of borrower systems and capacities and to longer-term goals of inclusion of minorities and poverty reduction. Evidence suggests that the communes with the 50 lowest poverty rates were more likely to be selected rather than the communes with the 50 highest poverty rates. The same trend is evident in the prioritization and grouping of selected communes for implementation sequencing.
There was also a misalignment of the disbursement schedules between the government system and the RWSS PforR which caused frictions in financial management. The contractor of the water supply schemes bore the cash deficiencies in the annual budgets of the PforR in the participating provinces, per the contract provision. The daily recording of the expenditures related to the PforR posed another challenge. During IEG’s interviews, respondents pointed out that the PforR could have benefited from stronger political buy-in of the central ministries with responsibilities for budget planning and allocation; that is, the Ministry of Finance and the Ministry of Planning and Investment.
Lessons
The IEG’s evaluation suggests the following lessons:
PforR design needs to be closely aligned with national policies and regulations, particularly regarding financial management. To avoid friction and imbalances in financial management, the design of PforRs needs to ensure that cash deficiencies do not occur in the provincial and subnational annual budgets because of differences among the disbursement schedules of the government system and the PforR operation. An exacerbating factor is that the Vietnamese budgeting and transfer system did not have a mechanism for adaptable performance-based transfers from the central government to provinces to enable results-based budget allocations. In 2020, the government of Vietnam also issued a decree (No. 56/2020/ND-CP) that specified that official development assistance and concessional loans be used to finance only capital expenditures and not recurrent expenditures, which prevented the development of a new PforR operation.
PforR design and implementation need to exercise equity and inclusivity in targeting beneficiaries to avoid selection bias against hard-to-reach ethnic and the poorest minorities and to reduce their vulnerability in the long term. Concrete measures need to be adopted to ensure that results-based operations provide implementing agencies with incentives to work in challenging areas by inlcuding, for example, the integration of poverty alleviation as a DLI target.
The existence of an enabling environment for private participation could enhance the effectiveness of PforRs. The results-based approach fosters the mobilization of private actors. The national policies and regulations promoted public-private partnerships, which were reflected in the design of the RWSS PforR. PforRs need to assess and harness opportunities to allow private enterprises to lead improvements in cost recovery ratios of water supply schemes. The experience with the RWSS PforR shows that Vietnam’s favorable enabling environment for private sector participation has enhanced the operation’s implementation effectiveness by improving the financial and operational sustainability of the country’s RWSS sector.
These lessons emerging from IEG’s evaluation can help guide World Bank support to institutions as they implement reforms and unlock the full potential of the private sector. Stay tuned for an overall assessment of the new financing instrument which will be presented through an overarching report that consolidates the findings of a cluster of Project Performance Assessment Reports on the Program-for-Results.
The full Project Assessment Report on the Results-Based Rural Water Supply and Sanitation Project in Vietnam can be found here.
Building inclusive, productive, and sustainable agrifood systems
Web Resource
Well-functioning agrifood systems play an important role in increasing food and nutrition security, reducing poverty, especially in low-income countries (LICs), and meeting climate and environment goals for sustainable development. They are especially important now in the face of rising prices and food insecurity. The World Bank Group uses a variety of instruments to help develop effective Show MoreWell-functioning agrifood systems play an important role in increasing food and nutrition security, reducing poverty, especially in low-income countries (LICs), and meeting climate and environment goals for sustainable development. They are especially important now in the face of rising prices and food insecurity. The World Bank Group uses a variety of instruments to help develop effective agrifood systems, so they are more productive, inclusive, and sustainable. A new evaluation from the Independent Evaluation Group (IEG) looks at a decade’s worth of Bank Group support for agrifood system development and identifies lessons for the future.
Agrifood systems comprise three components: the actors involved in the agriculture sector, the activities that these actors engage in, and the larger enabling environment. The actors cover the full range from farmers, agribusiness firms, processors, distributors to consumers. The enabling environment includes the policies, standards, and investments that affect sustainable production and market access.
IEG’s report finds that the Bank Group’s interventions in developing agrifood systems in the period between 2010 and 2020 were broadly relevant. But, gaps remain in scaling up and better targeting support to countries that need it the most. The report also found that the interventions were effective overall in improving the productivity, inclusion, and sustainability of agrifood systems. However, this was less so in LICs, particularly in West and Central Africa. This was partly because of the limited capacity, climate shocks, and other challenges that these countries face, especially those in fragile, conflict, and post-conflict situations. In addition, World Bank support for improving productivity was insufficiently diversified toward higher-value products that offer multiple benefits. The agribusiness investments of the Bank Group’s private sector arm, the International Finance Corporation (IFC), faced challenges in meeting environmental and social (E&S) standards, especially in LICs.
The report offers the following recommendations:
Combining production and market approaches: Production activities are poorly integrated with markets in many countries, especially LICs and countries at early stage of agrifood system development. Many LICs experience low agricultural productivity, which undercuts their efforts to reduce poverty and improve food security. Given their low productivity and limited access to markets and value chains, smallholders, and small producers in LICs remain poor and vulnerable to various shocks. Many of them struggle to shift from semi-subsistence agriculture to more market‐oriented agrifood enterprises.
The Bank Group can help reduce this weak market integration and fragmentation of smallholder production by exploring synergies between interventions that aim to support production activities with those that aim to support improved market access of producers. Interventions aimed at supporting production, i.e., on the supply-side, include improvements to technology, innovations including of digital technologies, delivery of inputs, and irrigation systems. Support for increasing market access, i.e., on the demand-side, includes identifying buyers, developing the needed market infrastructure (for example, storage and aggregation, logistics and cold chains) and facilitating value-chain linkages between smallholder farmers and small and medium enterprises (SMEs) with potential buyers in local, regional, and global markets. Access to finance is also key for supporting both production and marketing activities of farms and SME firms.
Such a mix of supply and demand-side interventions is particularly important for LICs and countries at early-stage of agrifood system development. The Bank Group can pursue such a combined approach through multiple avenues, including leveraging synergies across the Bank Group using parallel or sequenced interventions, through partnerships with other agencies, or through coordinated client actions.
Diversifying production and cultivating behavioral changes towards sustainable practices and standards: Where conditions allow, the Bank Group should support farmers and agribusiness firms in diversifying their production to also include high-value and more nutritious food products, such as fruit trees, vegetables, food legumes, fish, poultry, and livestock in addition to the traditional staples. Sustainable diversification should benefit smallholder farmers and SMEs, who often find it challenging to diversify their production or agribusiness to include high-value products. This will not only increase their agricultural productivity, but also provide nutritious foods that currently remain undersupplied or largely unaffordable to low-income consumers.
Supporting the successful production and marketing of such products will require the Bank Group to provide adequate financing and help producers pay attention to food safety and quality standards so that they can access competitive, regional and global markets.
At the same time, it will be important to ensure that the Bank Group also supports farmers and agribusiness firms in adopting sustainability practices. The Bank Group should encourage producers and value-chain actors to adopt climate-smart practices that use less resources, such as land and water, maintain biodiversity, and reduce environmental footprints.
Supporting Environmental and Social (E&S) Performance Standards in private sector investments: For private sector investments in agrifood systems supported by the IFC, the report found that when clients possessed the capacity and commitment to address E&S issues, or received support from IFC to do so, there was greater progress in improving their performance on E&S. Clients in LICs, especially, need assistance on recurring challenges (such as in wastewater management and occupational health and safety) and to support the implementation of E&S action plans and the Bank Group Environmental, Health, and Safety Guidelines.
IFC could support these clients through loan covenants, tailored advisory services, or blended finance.
The above lessons will help the World Bank Group better support its clients in developing agrifood systems that are more productive, inclusive, and sustainable and that can contribute to addressing the current global food crisis.
See also: Complementary Interventions for Agrifood System Development – Insights and Lessons
Measuring development outcomes of private sector investments: results, risks and outcome types
Web Resource
The private sector plays a critical role in promoting sustainable and inclusive development across the world. In view of its importance, a range of actors invest now in promoting the private sector in developing countries and regions. These actors span across the World Bank Group’s private sector arm, the International Finance Corporation (IFC), and risk guarantee arm, the Multilateral Investment Show MoreThe private sector plays a critical role in promoting sustainable and inclusive development across the world. In view of its importance, a range of actors invest now in promoting the private sector in developing countries and regions. These actors span across the World Bank Group’s private sector arm, the International Finance Corporation (IFC), and risk guarantee arm, the Multilateral Investment Guarantee Agency (MIGA) and a growing number of impact investors, along with a host of bilateral, private and multilateral organizations in between. One question they all face, is how to measure the development outcomes of their investments.
One approach adopted by both IFC and MIGA, and in some form by several other investors, is to identify expected development outcomes ahead of time, and use them as the basis for a framework to monitor and evaluate the progress and outcomes of investment projects. As a result of these various efforts, there has been increasing focus on improving the measurement of development outcome and the harmonization of the indicators and methods used, which would facilitate the comparison of outcomes. It has also led to a better understanding of the types of development outcomes and the key stakeholders who benefit from investments to support the private sector.
The development outcome types are as varied as the range of investors. One typical outcome involves increased access to goods and services. Other frequently found outcome types include improved quality of goods and services or enhanced capacity of private sector stakeholders. There is also a variety in the level and complexity of outcomes, such as improved living standards of individuals and increased productivity of enterprises, although these higher-level outcomes are not widely and systematically tracked for private sector investments. These would include higher-level outcomes at the sector and country level, such as increased competition, and the improved function and integration of markets. Yet despite their diversity, there is one trait they all share for the moment: little is known as to whether certain outcome types are hard to achieve, and if so, which ones?
An analysis of outcome types of IFC projects
The 2021 Results and Performance of the World Bank Group report included an analysis of the varying degrees of difficulty in achieving different outcome types. The main purpose of the analysis was to assess the relationship between certain outcome types and evaluation ratings and to determine what ratings actually reveal about the outcome types, and if higher ratings necessarily meant higher outcomes.
For IFC, the assessment focused on a group of IFC projects which were evaluated between 2012 and 2019, and to which the IFC’s newly created ex-ante and monitoring framework -AIMM (Anticipated Impact Measurement & Monitoring) system - was retroactively applied. The findings indicate that outcome types related to higher level, market outcome are, in general, harder to achieve compared to project level outcomes or harder to measure. This is because the achievement of market-level outcomes depends on the broader market environment and external factors such as the government actions. It is important to mention that in the analysis, the outcomes are only considered achieved if there is sufficient evidence to verify their achievements. In this regard, market outcomes are particularly hard to measure because of their longer-term nature, lack of good indicators, and the relatively smaller impact of one specific project. Among market outcomes, those related to enhanced competitiveness, integration, and sustainability in the market had lower achievement rates – and evidence for such outcome is hard to be obtained.
Among the outcome types which would impact directly on project stakeholders, outcome types related to increased access to goods and services, particularly those related to micro, small and medium enterprises (MSMEs), are relatively hard to achieve, compared with outcome types related to the quality of goods and services or increased linkage with suppliers and distributors. For example, increasing access to goods and services for MSMEs requires expanding lending to MSMEs, enabling them to borrow from financial institutions, which is not entirely within the control of the project.
Some limitations of the analysis.
The above results should be taken with certain caveats. The assessment was based on the projects to which the AIMM system was retroactively applied, meaning the rigor that the framework should bring might not have been present at project approval. In addition, IEG’ assessments of the achievements do not necessarily coincide with the monitoring results based on the AIMM system, where different timing between the monitoring and evaluation could be one of the reasons for the difference. Therefore, along with ongoing efforts to understand the varying challenges presented by different outcome types, it would also be helpful to continue the assessment of different degrees of achievements per outcome type, especially when projects with original AIMM assessments are evaluated.
Another important aspect to be considered is the relationship between achievements of outcomes and evaluation ratings. The RAP assessment showed that harder-to-achieve development outcomes are not necessarily accompanied by lower evaluation ratings in the IFC’s projects, because the evaluation ratings do not only take into account the achievement of development objectives but other factors, such as industry benchmarks and unintended outcomes (For more detail, please see the RAP.) In some other evaluation approaches, on the other hand, the achievement of higher development outcomes that are aligned with the original project objectives may have higher weights in assigning the rating. Therefore, the relationship between the achievement of development outcomes and evaluation rating needs further analyses and further refinement of evaluation systems may be warranted.
Lessons for future approaches
With the above caveats in mind, the outcome type analysis has several implications:
Factoring in risk: Understanding the different degree of achievements per outcome type would help in identifying the risks in achieving specific outcome types. If achieving access to finance for MSMEs proved to be more challenging, for example, there should be a recognition of higher risk of achieving such an outcome at project approval so that an informed decision can be taken, or mitigation actions taken. If an ex-ante scoring/rating tool incorporates risks for achieving outcomes, it would benefit from the information on the difficulty of achieving certain outcome types.
Ratings/scores for development outcomes: Currently, there are various approaches to assigning scores and ratings for the expected and actual development outcome. In the approaches taken by IFC’s AIMM system, the ratings and scores are based on the magnitude of a project’s contribution to address a country’s development challenges. For example, power generation capacity added by the project is assessed against the country’s power deficit. However, in most cases, rating or scoring mechanisms are not necessarily differentiated by the type of outcome and their level of challenges. For instance, expanding access to finance for MSMEs would not be assigned a higher weight than improving the quality of the access to finance for MSME. Adding these aspects into the rating/scoring mechanism might lead to ratings and scores that offer a more comprehensive picture of the outcomes assessed.
Enhancing results measurements: As mentioned before, the analysis of outcome achievement identified a higher share of outcomes for which properly measuring the development outcome is a challenge. This is particularly the case for market outcomes, where nearly a third of outcomes lacked proper evidence. Results measurements have been more advanced for outcome types which are considered more immediate or direct outcomes, such as increased direct employment. Further efforts are needed for the effective monitoring of higher-level outcomes, which are related to the living standards or productivity of intended beneficiaries as well as market level outcomes. The ongoing efforts promoted through AIMM in IFC is encouraging, as it identifies clear outcome claims and related indicators and monitor them. This is expected to enhance result measurement overall and allow better identification of different degrees of challenges to achieve certain development outcome types. In addition, further harmonization or alignment of ex-ante and monitoring frameworks among development finance institutions and impact investors would allow comparison and benchmarking of the results achieved by different players, further enabling better understanding of the nature of development outcomes and the impact of investments in the private sector.
Toward Productive, Inclusive, and Sustainable Farms and Agribusiness Firms
Web Resource
This report assesses the relevance and effectiveness of the World Bank Group’s support to its clients on helping them develop more productive, inclusive, and sustainable farms and agribusiness firms.
This report assesses the relevance and effectiveness of the World Bank Group’s support to its clients on helping them develop more productive, inclusive, and sustainable farms and agribusiness firms.
Reducing Disaster Risks from Natural Hazards
Web Resource
This evaluation focuses on the World Bank’s support for reducing disaster risks caused by natural hazards. Expanding disaster risk reduction plays a central role in achieving the World Bank’s goals for climate change adaptation and resilience.
This evaluation focuses on the World Bank’s support for reducing disaster risks caused by natural hazards. Expanding disaster risk reduction plays a central role in achieving the World Bank’s goals for climate change adaptation and resilience.
Evaluation of World Bank Support for Public Sector Transparency and Accountability: Anticorruption, Justice, and External Audit Functions (Approach Paper)
PDF file
Supporting client countries in building an open and accountable public sector that serves the needs of all citizens is core to economic development. Within any public sector, there are specific institutions or entities that are explicitly tasked with establishing, preserving, and improving the integrity of the public sector through the promotion of government system-wide transparency and Show MoreSupporting client countries in building an open and accountable public sector that serves the needs of all citizens is core to economic development. Within any public sector, there are specific institutions or entities that are explicitly tasked with establishing, preserving, and improving the integrity of the public sector through the promotion of government system-wide transparency and accountability. The Bank has a long history of helping countries increase transparency and accountability and designing and implementing anticorruption programs. This evaluation will assess the relevance and effectiveness of Bank support to client country central government institutions/entities that have an explicit mandate to promote the integrity, transparency, and accountability of the broader public sector. The effectiveness of World Bank support will be assessed through the prism of (a) improved capacity of country-relevant institutions to perform their functions as a result of World Bank assistance, and (b) evidence of the actual contribution to these institutions to, and their role in, promoting public sector transparency and accountability.