Highlights

The World Bank has been effective at improving natural resource management practices, but there is little attributable evidence that this has led to a reduction in natural resource degradation or the associated human vulnerability of resource users.

Most evaluated natural resource degradation and vulnerability projects were rated at least moderately satisfactory, but ratings reveal little about the effectiveness or sustainability of natural resource—and vulnerability-related outcomes. Only 10 percent of the closed sustainable land management projects mapped to the Sustainable Development Practice Group—and none of the closed Social Protection projects with sustainable land management components—provided attributable evidence of resource restoration. Although groundwater activities strengthened institutional capacity, little is known about their effects on intensity and groundwater use patterns and their impacts on resource depletion. Although many small-scale fisheries projects achieved their resource governance objectives, few assessed fish stock health or the welfare of fishing communities.

Natural resource degradation and vulnerability projects do not adequately identify, assess, and address heterogeneous effects on different subgroups of vulnerable resource users. Sustainable land management projects infrequently use metrics to measure the vulnerability of resource-dependent populations. Most closed groundwater projects reduced vulnerability by increasing water availability but did not indicate to whom benefits would accrue. These same projects did not include mechanisms to prevent overuse—and therefore reduce vulnerability—in the long run. Very few small-scale fisheries projects assessed the effects of improved resource governance on the welfare of fishing communities.

This chapter examines the effectiveness of World Bank projects that aim to address natural resource degradation, associated human vulnerability, and the nexus between them. The chapter includes a portfolio review and analysis of SLM, groundwater, and small-scale fisheries projects. The evaluable forest portfolio relevant to the evaluation scope was small (n = 8), and hence its effectiveness is not assessed as a stand-alone category in this chapter. However, projects with forest-related technical and financial instruments (for example, agroforestry, payments for environmental services) are included in the SLM portfolio and are assessed in chapter 4.1

The portfolio consists of 253 World Bank investment projects approved during the evaluation period (fiscal years 2009–19) with total financing of almost $33 billion. Of these projects, 117 (46 percent) were closed, and 78 (30 percent) were evaluated at the time of this evaluation. The SLM portfolio consists of 104 SLM projects mapped to the Sustainable Development Practice Group and 41 projects with SLM activities mapped to the Social Protection and Jobs Global Practice,2 55 projects with groundwater activities, and 53 projects with small-scale fisheries support (table 3.1). Most of the portfolio maps to four Global Practices: (i) Environment, Natural Resources, and Blue Economy; (ii) Agriculture and Food; (iii) Water; and (iv) Social Protection and Jobs. Half of it is in Sub-Saharan Africa (table 3.2).

The Independent Evaluation Group (IEG) rated most closed projects as moderately satisfactory or above, but the ratings reveal little about reduced degradation, sustainable natural resource management, or associated vulnerability outcomes. IEG rated 79 percent of closed projects moderately satisfactory or above in terms of achieving their stated objectives. However, the project development objectives that were measured reveal little about the projects’ contributions to reduced natural resource degradation or associated effects on the vulnerability of resource users. There are three reasons for this. First, most projects tracked progress using corporate indicators that measure outputs, not outcomes. Most project results frameworks lacked outcome indicators. Second, causal attribution is a challenge because few projects used rigorous testing and analysis (including comparison to a counterfactual) to isolate the effect of the World Bank’s technical interventions. Third, the lack of a consistently applied definition of SLM in projects complicates measurement across the portfolio. For example, SLM metrics used in Brazil refer to acts taken by landowners to register their land with the rural cadastre, whereas in Ethiopia, SLM refers to the physical restoration of degraded lands on a wide scale. Yet these projects—like most projects in the portfolio—use the same output-level indicators (number of hectares put under SLM or SLM practices).

Thus, this portfolio review and analysis was designed as an exploratory learning tool to distill lessons and map evidence. Because there has been no previous systematic assessment of World Bank operations exploring the link between NRDV, key functions of the portfolio review were to (i) identify the World Bank’s relevant portfolio, (ii) analyze the type and consistency of metrics applied to capture natural resource- and vulnerability-related outcomes, and (iii) assess the adequacy and effectiveness of the key intervention areas.

Sustainable Land Management

The World Bank defines SLM as a knowledge-based procedure that aims to integrate the management of land, water, biodiversity, and other environmental resources to meet human needs while sustaining ecosystem services and livelihoods. SLM interventions are expected to progressively result in the sustainable management of land and associated resources in a manner that meets current and future human needs while protecting the integrity of ecosystem services (World Bank 2006).

The World Bank approved 145 operations that included support for SLM, of which 59 were evaluated. The portfolio includes 104 projects mapped to the Sustainable Development Practice Group (40 closed) that aimed to achieve SLM through a mix of physical interventions; technical and institutional support; and policy, economic, and financial tools (table 3.1). Half of these Sustainable Development SLM projects applied a participatory watershed management approach at the catchment level.3 Other Sustainable Development SLM projects aimed to improve on-farm soil and water management for improved productivity but did not apply a watershed management approach. The SLM portfolio also includes 41 projects mapped to the Social Protection and Jobs Global Practice (15 closed) that provided cash, or in some cases food, for work to resource users to restore degraded land and resources, often as part of a social safety net program that provided financial transfers to poor households.4

Resource-Related Outcomes of the SLM Portfolio

Most Sustainable Development SLM projects report on the area put under SLM practices but not on the attributable, resource-related outcomes of the intervention. Only 10 percent of the closed Sustainable Development SLM projects provided evidence of soil and resource restoration attributable to the project through measurements of vegetative cover, species richness, biomass, organic matter, waterflow, and soil stability. Good practice examples include the following: (i) the China Sustainable Management and Biodiversity Conservation of the Lake Aibi Basin Project (2011–16), which adequately measured the increase in vegetative cover and income effects for farmers; (ii) the Second Ethiopia Sustainable Land Management Project (2014–19), which demonstrated both an increase in biomass in the intervention areas and an increase in the dry season base flow of sampled microwatersheds; (iii) the Senegal Sustainable Land Management Project (2010–13), which measured the increase in organic matter in the soil in targeted areas; and (iv) the Shandong Ecological Afforestation Project (2010–17; see box 3.1). An almost equal number of closed projects attempted to measure these effects but dropped the indicators by project close.5 The use of resource-related outcome indicators in Sustainable Development SLM projects has also declined (23 percent of closed Sustainable Development SLM projects included resource-related outcome indicators, but these were present in only 11 percent of the active results frameworks).

Although SLM interventions are expected to progressively result in enhanced land productivity to reduce vulnerability, most Sustainable Development SLM projects do not establish attributable links between restoration activities and land productivity outcomes. The conservation and restoration of land and the pursuant efforts to manage that land sustainably can increase soil fertility, fodder, water, and ecosystem services, which in turn can increase land, resource, and livestock productivity, creating opportunities for economic diversification. Realizing improved productivity is crucial for farmers and other land users in adopting SLM practices. In the Sustainable Development SLM portfolio, 22 percent of projects have indicators that measure land-related productivity (for example, average crop yields, milk production), but only 3 percent include both attributable resource-related and productivity-related indicators.6 Several projects reported on increased agricultural productivity in areas where SLM was adopted but did not establish links to the restoration activities. These links were not made evident even in phased SLM projects.7

Social Protection projects that provide cash or food for work for resource restoration are not ensuring that the physical restoration is leading to enhanced land and resource productivity, which are critical for vulnerability reduction in the long run. There were 41 Social Protection projects approved during the evaluation period that provided vulnerable resource users with cash or food for work to remediate degraded land (through a combination of irrigation and canal rehabilitation, afforestation, agroforestry, soil conservation, and land restoration). Although half of the projects included an output-based indicator (number of hectares put under improved SLM practices), none measured the quality of the restoration effects, including the longest-running and most evaluated program in the Social Protection SLM portfolio in Ethiopia. Case analysis conducted in Ethiopia suggests that more attention should be paid to the quality of land remediation in Social Protection projects with SLM components (box 3.2).

Vulnerability-Related Outcomes of SLM Projects

In the absence of adequate targeting, it is not possible to determine the heterogeneous effects of SLM projects on different subgroups of vulnerable resource users. In the SLM portfolio, there is a tendency to identify project beneficiaries through geographic (for example, watershed) and political-administrative boundaries rather than identifying marginalized groups that are dependent on the degraded resources. Although this geographic targeting displays awareness of poverty-resource links, it does not adequately consider factors that prevent vulnerable resource-dependent subgroups—distinguished by income or wealth, risk-coping abilities, gender, race and ethnicity, and so forth—from actively participating in or benefiting from resource restoration activities. Crop farmers, including those who also rear livestock, are almost always cited as beneficiaries, but apart from indigenous groups in Latin America and the Caribbean and scheduled tribes in India, other groups such as pastoralists (in Africa), traditional communities, Afro-descendants (Latin America and the Caribbean), and the landless are rarely integrated into project design. As expected, therefore, few projects measure the disaggregated effects of SLM activities on different subgroups. Most striking is the finding that of the 21 SLM projects that list pastoralists as beneficiaries, only two measured their benefits (for example, access to pastoral infrastructure).

SLM projects that use cash for work have achieved short-term vulnerability-reducing effects, but the distribution and magnitude of these effects is not known. Half of the 104 Sustainable Development SLM projects and all Social Protection SLM projects included a form of payment for work for the restoration of degraded land and resources. In all cases, the cash-for-work programs were implemented in degraded areas with high poverty and resource dependency. In these areas, low wages ensured that vulnerable people were targeted. What is not known, however, is how the work was distributed or the magnitude of the vulnerability-reducing effects. There are three reasons for this. First, although Social Protection SLM projects report on the number of persons who worked or the total wages paid, most Sustainable Development SLM projects do not, rendering impossible an assessment of vulnerability reduction. Second, it is not clear who participated in the cash-for-work programs. Projects rarely included gender, youth, or other social inclusion considerations. Nor is it known whether workers were local, to strengthen the synergy between restoration activities and reduced vulnerability. Third, in many cases, projects do not indicate who owns or has rights to the land being restored.8 For example, for programs conducted on communal land, projects did not indicate how access and use rights would be preserved after project close (for example, how grazing rights would be maintained after tree planting).

Gender issues are almost always cited in SLM projects, but gender is often treated as a homogenous category, disregarding differences in voice and agency that influence access and SLM benefits. Ninety-three percent of SLM projects cite gender-related SLM issues, and 75 percent include gender-disaggregated indicators. Yet projects mainly report on the number of women reached, trained, or who have adopted practices, not on which women benefited and how. This is important because widowed and unmarried women often have less access to land and tend to benefit less from SLM activities. Unequal land and resource access were cited in one-third of the projects but rarely addressed. Good examples of gender-integrated SLM outcomes include women’s acquisition of land certificates or an increased share of commercial crop production.

Furthermore, the full social and environmental effects are unknown for several closed projects that overachieved their SLM land area targets. The introduction of new land use regimes can cause tensions among resource users with competing claims or historic grievances if these risks are not identified and mitigated. Of the 40 closed Sustainable Development SLM projects, 23 supported the adoption of SLM practices on more land than was planned. The widest outlier was a project in Burkina Faso that put 213,320 hectares of land under SLM practices against a target of just 15,000 hectares.9 But although the SCDs or CPFs for two-thirds of the countries in which these activities took place cite potential resource-related conflict risks, none of these projects with expanded activities restructured or conducted expanded social or environmental analyses. From an environmental perspective, restructuring would also need to have considered net agroecological trade-offs (for example, the effect that tree planting upstream has on soil erosion and water availability downstream) in the targeted areas.

However, the active SLM portfolio has significantly improved in its treatment of social and conflict risks exacerbated by land and resource degradation. More than half of active SLM projects in countries with resource-related conflict risks identified by the SCD or CPF included explicit conflict management mechanisms (for example, local conflict identification and reduction capacity, trust-building arrangements, and resolution and mediation processes). These include projects in countries where conflict mechanisms were not included in a prior phase. A good example is the Ethiopia Lowlands Livelihood Resilience Project (approved in 2019) that aims to put 700,000 hectares of land under SLM and includes mechanisms for managing conflict, such as financing activities to understand the root causes of conflict and to resolve conflict.

Nexus Outcomes of the SLM Portfolio

No SLM project adequately assesses the nexus between resource-related outcomes and the vulnerability reduction of relevant resource users. Only 10 percent of the closed Sustainable Development projects and no Social Protection project adequately provided attributable resource-related evidence. Only two closed Sustainable Development projects tried to measure the link between resource restoration and human vulnerability, but in these cases, the analysis was incomplete.10

Sustainable Groundwater Management

The World Bank approved 55 lending operations with groundwater components over the evaluation period, of which 28 were closed.11 Lodged within larger water projects (for example, irrigation and water supply), the groundwater interventions include support for institutional strengthening; data gathering and monitoring; water availability and quality; and, to a lesser extent, policy, legal, and regulatory reforms. Many of these projects are in the South Asia and the Middle East and North Africa Regions, where aquifers are increasingly being depleted for agriculture and other uses (table 3.2).

Resource-Related Outcomes of the Groundwater Portfolio

Most groundwater management activities in World Bank projects have been satisfactorily implemented, but effects on groundwater use patterns and the status of groundwater depletion is not known. About 90 percent of institutional strengthening activities in groundwater projects were implemented. The World Bank helped the governments of China, Mexico, and Zambia develop groundwater and aquifer management plans and is currently helping India to build a comprehensive national water resources information system. With World Bank assistance, groundwater from key aquifers in Morocco is now being regulated to promote equitable sharing among users. In Brazil and Mongolia, new systems have been established to enable key groundwater allocation decisions. However, little is known about the effects of these activities on intensity and patterns of use or their impacts on resource depletion. This is because reliable and comparable data on groundwater stock, flow, and quality are not available. This challenge, identified in a World Bank groundwater study (Wijnen et al. 2012), continues to undermine efforts to help clients make critical groundwater use decisions.

Sustainable groundwater management has also been challenged by political economy constraints, especially in the Middle East and North Africa. More than half of the groundwater data generation and monitoring activities were not achieved, with constraints observed in the Middle East and North Africa. For example, in Tunisia, project reporting states that “vested interest provided powerful incentives not to publicly disclose the exact groundwater extraction of larger farmers . . . subverting foundations for robust groundwater management” (World Bank 2015). World Bank support for groundwater reform in Jordan, by contrast, provides a good example of how technical assistance (for example, mapping and remote-sensing technologies) can support reforms (box 3.3). These projects were implemented during the Arab Spring uprisings of 2010–12, when many reforms were affected by political and social unrest.

Vulnerability-Related Outcomes of the Groundwater Portfolio

Although most closed groundwater projects increased water availability, they did not articulate how the vulnerable would gain access. Most closed groundwater projects indicated that everyone living in the project area would benefit from the project interventions. Only one-third of the portfolio aimed to ensure that vulnerable groups (for example, poor rural households, farmers below the poverty line) would benefit from the groundwater intervention. A good example is the India Assam Agricultural Competitiveness project, which ensured that water use benefits accrued to the landless. There is strong integration of gender-disaggregated indicators in the portfolio, but only one-fifth of closed projects had gender-sensitive designs (for example, women represented in decision-making roles in a water user association).

Very few closed groundwater projects that increased water availability include mechanisms to prevent overexploitation that threatens vulnerability in the long run. Almost two-thirds of closed projects with groundwater components improved water availability through irrigation efficiency and increased water supply. Yet only eight of these projects, or 30 percent of the closed portfolio, include regulations and measures to reduce pressure on stressed groundwater systems.

Nexus Outcomes of the Groundwater Portfolio

Although one-quarter of the closed groundwater portfolio was designed to achieve NRDV nexus outcomes, most projects struggled to find a balance between increasing groundwater availability and achieving sustainable use. Seven of the 28 closed groundwater projects included mechanisms to reduce the vulnerability of water users by improving water availability or quality, while also putting in place regulations and instruments to curb overextraction and unsustainable use.12 Most other projects increased water availability or put in place measures to improve water quality but did not put in place regulations or mechanisms to curb depletion.13

Sustainable Small-Scale Fisheries Management

The World Bank approved 53 projects with small-scale fisheries activities, of which 28 were closed and evaluated. Almost 70 percent of the project development outcomes supported governance and institutional capacity building, including support for community participation and associated vulnerability reduction.

Resource-Related Outcomes of the Small-Scale Fisheries Portfolio

Two-thirds of the small-scale fisheries portfolio, which mainly focused on fisheries governance to address overexploitation and resource depletion, was rated moderately satisfactory or above. The West Africa Regional Fisheries Program supported fisheries management and governance interventions in Cabo Verde, Ghana, Guinea-Bissau, Liberia, Senegal, and Sierra Leone. The program helped many of these governments improve the monitoring and enforcement of inshore exclusion zones.14 Satellite-based monitoring systems were established for large-scale vessels, increasing patrols and, in some cases, prosecution of infractions. The highest-funded project in the program, in Ghana, however, failed to achieve its aim of reducing external fishing pressure and overexploitation from large-scale fishers. The large-scale fleet increased, inactive vessels were not deleted from the registry, and closed seasons—seasons when fishing is prohibited—were partially implemented based on the government’s fisheries management plan. Interventions to establish comanagement arrangements in the coastal fisheries sector also did not advance as envisioned.

However, few projects assessed and addressed overall fish stock health. Of the 28 closed projects, 5 were designed to address and measure the effects of project activities on fish stock health. Notably, Indonesia’s Second Coral Reef Rehabilitation and Management Program (2004–12) achieved a 29 percent increase in the reef fish population. But other programs, including in Mindanao (in the Philippines), saw decreases in fish biomass (due to an unwillingness to enlarge the size of marine sanctuaries, continued poaching, and dynamite fishing), despite efforts to reverse this trend.

Vulnerability-Related Outcomes of the Small-Scale Fisheries Portfolio

Few small-scale fisheries projects focused on and measured the vulnerability-reducing effects of improved small-scale fisheries management and governance. Six closed small-scale fisheries projects addressed and measured the vulnerability-reducing effects of project-supported management and governance reforms. This mostly occurred in East Asia. For example, in Indonesia’s Second Coral Reef Rehabilitation and Management Program, mechanisms put in place to empower and incentivize fishing communities through comanagement arrangements were legally recognized and enforced. These interventions have, in turn, resulted in fewer fishing violations and increased incomes for fishing communities. The Vietnam Coastal Sustainable Resources for Development Project (2012–19) helped establish and expand comanagement partnerships with fishing communities, which contributed to a 30 percent reduction of violations within the local fishing communities and increased incomes. The project sites provided useful lessons and contributed to the amendment of the Fisheries Law (2017), which introduced comanagement and rights allocation to communities countrywide.

Nexus Outcomes of the Small-Scale Fisheries Portfolio

Small-scale fisheries nexus outcomes were achieved only in a single project. The Indonesia Second Coral Reef Rehabilitation and Management Program is the only example of a small-scale fisheries project that measurably and effectively achieved NRDV aims. It both improved fish stock health and enabled an increase in average incomes in the project area. Small-scale fisheries projects in the Philippines and Senegal were designed to achieve NRDV nexus outcomes, but both had unsatisfactory outcomes.