Highlights

At the country level, results frameworks do not capture the World Bank’s contribution to conflict-related country outcomes well. This reflects the absence of both a clear conflict narrative and an integration of conflict-related issues into country objectives.

Few Country Partnership Framework results frameworks are adaptive and capture conflict-reduction aims; the World Bank’s reliance on quantitative metrics, attribution, and short time frames may not suit the nature of these programs and their contribution to higher-order outcomes. Country Partnership Frameworks that have received additional fragility, conflict, and violence International Development Association allocations have a more coherent narrative about their transition. Yet their results frameworks only monitor the progress of allocation areas directly supported by the World Bank.

An accurate picture of project outcomes is elusive in conflict-affected countries because only a small share of investment projects are evaluated. Evaluations and validations of investment operations in conflict-affected areas are not comprehensively assessing Bank performance. Many trust-funded activities, which are often used in these contexts, are not being evaluated by the World Bank to support adaptive decision-making and learning; they also fall below the threshold for Independent Evaluation Group validation. Expanding the share of projects in conflict situations that are evaluated and validated and revising evaluation guidance would provide a more accurate picture of outcomes and contribute to learning.

There are information gaps about the way that the World Bank is monitoring or assessing unintended outcomes in conflict-affected areas. Little is known about the extent to which World Bank operations in conflict-affected areas may be exacerbating underlying grievances. Relatedly, although attention to gender-based violence by the World Bank is increasing, the percentage of at-risk projects in conflict-affected areas that report on mitigation measures remains low and is inconsistent. Although the use of armed security personnel is rising, few projects indicate how associated risks will be mitigated in project areas.

Measuring Project-Level Results in Conflict-Affected Countries

Many projects in conflict-affected countries are not achieving their development objectives as articulated in project results frameworks. For the total evaluated portfolio of 171 operations, 44 percent of objectives in DPFs and 34 percent of IPFs were rated moderately unsatisfactory or unsatisfactory by IEG (figure 5.1). This is in contrast to all countries on the FCS list (that is, fragile countries, regardless of their conflict status): just 26 percent of IPF and 32 percent of DPF in those countries had moderately unsatisfactory or lower outcomes, ratings that correspond more closely to the average for all World Bank–borrowing countries.

Figure 5.1. Implementation Completion and Results Report Review Outcome Ratings across Country Groupings, FY10–20

Image

Source: Independent Evaluation Group.

Note: A total of 1,208 World Bank operations approved during FY10–20 were evaluated (to end November 2020), 361 were in countries on the FCS list, and 171 were in conflict-affected countries. DPF = development policy financing; FCS = fragile and conflict-affected situation; FY = fiscal year; IPF = investment project financing.

Share Tweet Facebook WhatsApp Linkedin Email

Figure 5.1. Implementation Completion and Results Report Review Outcome Ratings across Country Groupings, FY10–20

Source: Independent Evaluation Group.

Note: A total of 1,208 World Bank operations approved during FY10–20 were evaluated (to end November 2020), 361 were in countries on the FCS list, and 171 were in conflict-affected countries. DPF = development policy financing; FCS = fragile and conflict-affected situation; FY = fiscal year; IPF = investment project financing.

Complicating the measurement of project outcomes in conflict-affected countries (besides the obvious challenges of constraints on data collection) is the fact that many of these projects are not being evaluated or validated. Programmatic trust funds are often used in such contexts. More than half of all operations used trust fund financing, and more than one-third of all projects exclusively used trust fund financing (particularly in nonmember states in arrears, such as Somalia and Sudan), from 13 development partners through 211 trust funds. Lending operations supported by trust funds, even those that are small, should adhere to reporting requirements, including ICRs.1 However, 50 percent of all closed projects in the portfolio did not have ICRs. Also, since IEG only validates projects with an ICR that are financed at a level of $5 million or more, only 37 percent of all closed projects were validated, compared with 52 percent in all IDA or blend countries (figure 5.2).

Figure 5.2. Evaluations and Validations of Projects Closed during FY10–20 in Conflict-Affected Countries, by Country Grouping

Image

Source: Independent Evaluation Group.

Note: FY = fiscal year; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; ICR = Implementation Completion and Results Report; ICRR = Implementation Completion and Results Report Review.

Share Tweet Facebook WhatsApp Linkedin Email

Figure 5.2. Evaluations and Validations of Projects Closed during FY10–20 in Conflict-Affected Countries, by Country Grouping

Source: Independent Evaluation Group.

Note: FY = fiscal year; IBRD = International Bank for Reconstruction and Development; IDA = International Development Association; ICR = Implementation Completion and Results Report; ICRR = Implementation Completion and Results Report Review.

Expanding the share of projects rated by IEG would provide a more accurate picture of project outcomes in conflict-affected areas, including by likely increasing the percentage of projects achieving their development objectives. Nearly all nonvalidated operations have been rated moderately satisfactory or higher by the World Bank. If past ratings are used as a proxy, then IEG would downgrade only 10 percent of these from a moderately satisfactory or higher rating to moderately unsatisfactory or lower. This would bring average performance more in line with all projects in FCS. It would also allow for a more accurate assessment of the type of outcomes achieved through targeted projects that are purposively financed at smaller levels in line with capacity constraints, to minimize risks, to pilot and innovate, or to address development challenges related to justice, governance, or other economic or social issues.

Evaluations and validations of investment operations in conflict-affected areas are not comprehensively assessing Bank performance. In a conflict situation, although an operation may fail to achieve its development objective, the World Bank may nevertheless have performed well, for example, by setting ambitious yet feasible targets; ensuring that design and implementation are informed by appropriate risk identification and mitigation; and engaging in risk-informed adaptive management. This real-world disconnect between outcomes and Bank performance often reflects the high degree of uncertainty that typifies conflict situations. However, IEG found that assessments (i) tend to focus on the overall outcome (that is, achievement of outcomes relative to objectives and targets set at the design stage), (ii) lack references to conflict in the Bank performance section, and or (iii) assign Bank performance a low rating because of the project outcome. For example, there is no reference to conflict in the Bank performance sections of half of all IPF validations, and there is only one ICRR that links conflict considerations to a Bank performance rating of moderately satisfactory (for taking into account past lessons, hiring the right team, and integrating risk identification and mitigation into design and implementation).

There is also a significant difference between the way the World Bank and IEG observe and assess the success of DPOs in conflict-affected situations, with IEG systematically downgrading DPOs to the moderately unsatisfactory (or lower) rating. DPOs in conflict-affected countries were downgraded at the same rate as DPOs in the broader World Bank universe: 31 percent and 32 percent, respectively. However, in several DPOs in conflict-affected countries, an acute deterioration in operating conditions during program implementation negatively affected clients’ ability to support ongoing reform programs, which then led to worse program outcomes in evaluations when compared with DPOs in nonconflict situations. Although the outcome ratings assigned by operations reflected these less-than-satisfactory outcomes, IEG downgraded these lower-performing project ratings even further. Of the 55 DPFs with ICRRs,2 nearly one-third of operations were downgraded by IEG. However, unlike for DPOs from non-conflict-affected countries, where the 59 percent of downgrades were from highly satisfactory or satisfactory to moderately satisfactory, fully 94 percent of downgrades in the conflict-affected universe were to ratings of moderately unsatisfactory or lower (including two from satisfactory to moderately unsatisfactory, eight from moderately satisfactory to moderately unsatisfactory, and six from moderately unsatisfactory to unsatisfactory). Reasons cited by IEG for the downgrades include lack of achievement of outcomes and overambitious design, with ICRRs for about half of DPOs mentioning conflict in the downgrade explanation.3 IEG is currently working with World Bank management to revise DPO ICR and ICRR methodologies to ensure a common framework that allows evaluations to better capture the important nuances of World Bank engagement, including in situations of conflict.

Higher-Level Outcomes at the Country Level

Results frameworks in conflict-affected countries do not capture the World Bank’s contribution to country outcomes well, paralleling IEG’s earlier finding from the outcome orientation at the country-level evaluation that noted this for the broader World Bank portfolio. For most conflict-affected countries, information on the progress of the country’s transition out of fragility needs to be obtained from country progress discussions that also rely on standard results tracking as part of the CPF cycle. However, the World Bank’s current reliance on metrics, attribution, and short time-boundedness does not suit the nature of country programs in conflict-affected countries. Although country teams in some of the evaluation universe were found to practice adaptive management, the World Bank’s static country-level results system does not effectively support them in doing so. The outcome orientation evaluation recommended a more decentralized country-level results system to help country teams shape their monitoring and evaluation plans in line with the country’s own adaptive management and learning needs.

Only governance and state effectiveness pillars in CPF results frameworks in conflict-affected countries have a clear conflict (or fragility) narrative; other pillars remain “business as usual.” Governance pillars often articulate the link among state legitimacy, institutional effectiveness, and stability; they also include relevant indicators related to institutional transparency and accountability (for instance, budget transparency and revenue transfers) and citizen engagement. Pillars in the sustainable and human development sectors, however, tend to use traditional sector narratives and output indicators devoid of links to conflict reduction. Overall, CPFs have not reported on citizen feedback and grievance issues emanating from project monitoring and evaluation systems; these tools can be used to track conflict considerations at the country level (except for the Afghanistan CPF, which included a pillar on citizen engagement). In countries where displacement is a development challenge, and that have projects that address displacement, CPFs have not consistently captured displacement-related aims. CPFs have also not reported results geographically, even though many programs target aid subnationally to address conflict drivers, including inequality.

CPFs for conflict-affected countries that have received IDA FCV allocations (such as the TAR/TAA) are presenting a coherent narrative about their transition out of fragility and are mainstreaming conflict considerations in their country portfolio. A successor to the TAR introduced in 17th Replenishment of IDA, the TAR/TAA provides critical financing for countries emerging from conflict (or social or political crisis or disengagement) and where there is a window of opportunity to pursue reforms that can accelerate a transition out of fragility and build resilience.4 So far, the Board has approved eligibility for the Central African Republic, Madagascar, Somalia and Sudan5 after having established government commitments and agreed on milestones with the borrower and the World Bank.6 Pursuant to these allocations, Madagascar’s FY17–21 CPF was fully conflict sensitive. Similarly, the Central African Republic’s CPF includes frank references to drivers of conflict and as such relevantly focuses on regional and social disparities, elite capture, and weak governance. To support implementation, the World Bank has increased its staff presence. Overall, the FCV IDA allocations—and the ensuing way that CPFs are written—have enabled a franker dialogue among CMUs, clients, and the Board.

Although a notable first step, the TAA process could be improved by encouraging even greater inclusion within subsequent CPFs of conflict-sensitive indicators. TAA Eligibility Notes contain lofty milestones for ambitious reforms, some of which lie (by definition) outside the scope of the World Bank’s mandate, but parallel CPFs do not fully embrace a conflict-sensitive approach, since their results frameworks do not consistently include conflict-sensitive indicators. The inclusion of conflict-sensitive indicators would ensure that the World Bank is not only weaving a conflict narrative into its CPF but also prioritizing and monitoring those areas and actions that do lie within the World Bank’s mandate. For instance, although the Central African Republic CPF has mainstreamed conflict issues, its results framework does not track citizen feedback about service delivery quality or associated perceptions of state legitimacy, nor does it track land- and resource-related grievances, even though the country program heavily supports the development of traditional value chains (mining, forestry, select cash crops). In the Central African Republic, access to justice is also a major theme, and some CPF indicators track progress against disarmament, demobilization, and reintegration objectives, but there is no tracking of IDPs or host communities. A contrasting example is the Madagascar CPF, which adeptly includes conflict-sensitive indicators on decentralization, increased revenue mobilization, and the fight against rent capture, three critical agenda items within the TAA to facilitate the country’s escape out of the fragility trap (and that are within the World Bank’s mandate). However, like the Central African Republic CPF, it does not include indicators to track citizen feedback on the quality of service delivery or citizen perceptions on government inclusion and responsiveness.

Contribution of IPF to Country Outcomes

CPF results frameworks need to integrate information collected from projects focused on reducing the drivers of conflict, but projects are not reporting on these aims. Rather, most operations that include an objective or a theory to promote cohesion or stabilization assessed the same technical and corporate level output indicators as those operating outside of conflict contexts. Half of IPF projects assessed (n = 94 across five Global Practices) included explicit conflict-related aims in their project development objectives (8 percent) or project theory (43 percent). About 70 percent of these projects were approved from FY15 onward. The most frequently occurring aims are (i) social cohesion between groups (horizontal) and between citizen and state (vertical); (ii) peace building and stabilization; and (iii) reduced resource-related conflict (table 5.1). However, of the 12 closed projects with articulated conflict-related aims, only 3 ICRs reported on these aims. For example, the Afghanistan National Solidarity Program III (FY10–15) measured its contribution to building democratic and legitimate local governance processes using proxy indicators and through impact evaluations and beneficiary surveys. The Central African Republic Service Delivery and Support to Communities Affected by Displacement Project (FY17) measured perceptions of how peaceful coexistence activities benefited communities.

Social Cohesion

By far, the most common conflict-related outcome in IPFs is social cohesion, but this outcome is difficult to achieve at the project level, and even more difficult to measure. Many projects operating in conflict situations seek to strengthen social cohesion, both horizontal (within and between conflict-affected individuals and groups) and vertical (between citizen and state; see table 5.1). Community-driven development investments have especially attempted, through various implementation mechanisms, to repair group bonds frayed by war and violence. Although these efforts are of merit, they have not been measured.7 The World Bank is also using social contract diagnostics to assess what citizens expect from the state and what the state can legitimately expect from citizens. IEG’s Social Contracts and World Bank Country Engagements evaluation (World Bank 2019f) showed that these efforts are most effective when anchored in regional and country engagements and focused on social contract renewal, as was the case in the Middle East and North Africa after the Arab Spring. Even so, in several Middle East and North Africa cases, citizen engagement–specific operations were missing from the portfolio (especially in Iraq, Lebanon, and the Republic of Yemen). In the absence of this anchoring—of social country diagnostics in CPFs and the portfolio (especially regarding service delivery)—project efforts are unlikely to have a measurable impact on the social contract.

Use of Labor-Intensive Public Works to Support Political Stabilization and Peace-Building Goals

There is a need to test assumptions about the contribution of labor-intensive public works to wider stabilization and peace-building efforts in different conflict-affected settings.8 Labor-intensive public works (LIPW) was the most frequently cited modality used to support stabilization aims (among many other proactive measures), including in Afghanistan, the Central African Republic, the Democratic Republic of Congo, Niger, Nigeria, and the Republic of Yemen. LIPW are used to increase state presence and dampen grievances, and, in some cases, they are intended to be used as a tool to dissuade potential fighters (for example, youth) from joining insurgent groups. For example, the 2014 Democratic Republic of Congo Eastern Recovery Project is using LIPW to prevent youth in high-risk zones from being recruited into armed groups. Although the World Bank indicates that short-term employment programs are only one small piece in working toward longer-term stabilization, the evidence that exists on the links between LIPW and stabilization goals shows only modest effects (Mvukiyehe 2018). As such, these theories and activities should be further tested and refined.

Reducing Resource-Related Conflict

Several projects in the assessed portfolio seek to reduce resource-related conflict, but only 45 percent of those projects deployed relevant conflict-related activities, and even so, envisioned outcomes were not discussed in the CPFs. Farmer-pastoral conflict over scarce resources contributes to instability in many conflict-affected countries, including Burkina Faso, Cameroon, Chad, Mali, and Nigeria. The aim of reducing intergroup resource-related conflict was articulated in agricultural projects in Africa and was addressed through mechanisms such as community-based natural resource management practices. For example, the Chad Emergency Food and Livestock Crisis Response Project sought to prevent agropastoral conflict by demarcating 250 kilometers of transhumance corridors, establishing committees of elders, and organizing peaceful coexistence forums. Yet none of the projects that sought to reduce resource-related conflict reported on conflict outcomes, and as such, these outcomes were not tracked in CPFs.

Contribution of DPF to Country Outcomes

DPFs in conflict-affected situations have contributed to critical country-level outcomes not captured by results frameworks or indicators. Although DPO results frameworks cannot capture the entirety of program impact, this is particularly the case in conflict-affected situations. DPOs failing to achieve program results—for instance, because of a deterioration in the stability of a conflict-affected country (the main factor found to negatively affect DPO outcomes in such countries)—may have nonetheless enabled critical outcomes outside of the parameters of formal results frameworks. Even when conflict negatively affects outcomes, the engagement can contribute to positive impact, often in ways that are difficult to capture in results frameworks. Also, DPOs can contribute to higher-level outcomes that fall outside their short program horizons.

Fiscal Stabilization in the Context of Potential Collapse

Some DPOs in conflict-affected countries have contributed to the higher-order outcome of fiscal stabilization. This is often the result of the budget support provided, which can support the maintenance of core public services and reduce reliance on more expensive or destabilizing sources of finance. In such situations, sometimes just “keeping the lights on” or simply maintaining development gains (that is, preventing slippage) can be considered a success (Kelly, Nogueira-Budny, and Chelsky 2020). For example, in 2017, the World Bank’s DPF to the beleaguered government of Chad was unsuccessful from a results perspective. However, that same engagement succeeded in shoring up a government teetering on the verge of fiscal collapse, an outcome not captured within the project’s results framework; an analogous situation occurred in Iraq, in the face of conflict with the Islamic State (box 5.1). The reforms supported by DPF can also contribute to fiscal stabilization to the extent that they are successful in, for example, addressing constraints on private sector development or improving the management of public finances.

Strengthening Country Systems

Some DPOs have contributed to the higher-order outcome of building and strengthening core country-level fiscal institutions by using national administrative and public financial management systems. Many development partners halt general budget financing after deteriorations in a country’s stability because of concerns over political and fiduciary risks in inchoate contexts. The 2011 World Development Report warned about the pitfalls of budget support in fragile states, recommending financial instruments that support the tracking of expenditures (for example, IPF, with its clear relationship between disbursements and expenditure; World Bank 2011). However, the provision of budget support has a positive impact on public financial management and accountability systems by channeling resources through country systems and supporting budget planning and execution processes (OECD 2005). In using country systems and subjecting financing to the client’s own implementation processes, DPOs avoid the construction of parallel systems—including political decision-making mechanisms—that can undermine the strength and legitimacy of government (Dollar and Pritchett 1998).

Signaling

Some DPOs in conflict-affected countries were found to have contributed to higher-order outcomes by “signaling” to others that a country’s political stability, macroeconomic framework, or fiduciary safeguards had improved, so that other development partners found it safe to (re)engage. Although geopolitics and the politics of the World Bank donor countries generally drive the availability of World Bank budget support, the presence of DPOs does signify progress. Given the instrument’s prerequisite of the presence of a stable macroeconomic policy framework and ability to ensure fiduciary controls, the implementation of a DPF signals to others that a client has reestablished the controls necessary to agree to a reform program that makes it a worthwhile recipient of additional development support (box 5.2).

Measuring Unintended Outcomes

Do No Harm

Although it is critical to ensure the “do no harm” principle, little is known about the extent to which World Bank operations in conflict-affected areas are exacerbating underlying grievances. The World Bank’s FCV strategy notes that activities financed by donors and development partners should avoid causing or contributing to adverse human or environmental impacts (World Bank 2020f, 17). In conflict-affected contexts, this calls for them to adopt a context-specific, conflict-sensitive approach based on adequate due diligence, diagnostics, risk analysis, and citizen engagement. A systematic review of the causal impact of development aid (including that of the World Bank) on violence in countries affected by civil war found that aid in conflict zones is more likely to exacerbate violence than to dampen it: a violence-increasing effect occurs when aid is misappropriated by violent actors or when violent actors sabotage aid projects to disrupt cooperation between the local population and the government.9 In rare instances of World Bank projects where this is measured, there are examples of increased violence by insurgents despite the projects being rated satisfactory. This was the case in the World Bank–supported conditional cash transfer program Familias en Accion in Colombia. Although the project achieved satisfactory marks for school attendance and food consumption, in some municipalities, the project subsidies incentivized information sharing with the government, increasing its territorial control and encouraging the Revolutionary Armed Forces of Colombia to commit more indiscriminate acts of violence to recapture lost territory.

Project size can also lead to unintended outcomes, with some evidence showing that large programs can contribute to conflict. Two recent papers summarizing the aid experience in Afghanistan and Iraq found that program size can be linked to conflict. In Afghanistan, large-scale operations were found to be more likely than smaller ones to create unrealistic expectations and were more subject to corruption and targeting by insurgents (Iyengar, Shapiro, and Hegarty 2017).10 In Iraq, research shows that smaller projects that were coordinated with security forces were more successful (Berman, Shapiro, and Felter 2011). More research on this topic can help donors design such interventions in ways that improve livelihoods while lowering the risk of exacerbating conflict.

Gender-Based Violence

Attention to GBV is increasing, but many at-risk projects still lack mitigation measures. Across the sectors analyzed, 28 percent of projects articulated how they would address GBV in project areas. Seventy percent of these projects were approved after the 2018 introduction of a Good Practice Note for Addressing Gender-Based Violence in Investment Project Financing involving Major Civil Works and the rollout of the new Environmental and Social Framework. Projects that identify GBV risks and that have GBV prevention mechanisms address GBV in bidding documents, assessments, and agreed codes of conduct. They include GBV focal points within grievance redress mechanisms, awareness-raising activities, and accounting for costs of GBV mitigation actions. Good examples include the Burundi Infrastructure Resilience Emergency Project, which had all project workers sign a code of conduct in the local language, and the South Sudan Safety Net and Skills Development, which raised awareness on GBV and used GBV focal points in its grievance redress mechanism. However, there is uneven treatment across at-risk projects, within Global Practices, located in conflict-affected areas (for example, those that may induce a population influx; those that target IDPs or refugees [World Bank 2020f]; or those that provide cash transfers to women [Roy et al. 2017]). For example, there are a few good practice transport projects that identify and mitigate the GBV risks associated with population influx, such as the 2019 Niger Rural Mobility and Connectivity Project, yet most do not, including projects that operate in similar risk areas and implement similar activities, such as the 2018 Mali Rural Mobility and Connectivity Project. Social protection projects that transfer cash to women in conflict areas can exacerbate interpartner violence. Forty percent of the social protection projects addressed GBV risks, such as the 2016 Nigeria National Social Safety Nets Project, whereas a similar project implemented in a similar context—the 2016 Mali Adaptive Social Safety Nets Project—did not.

Mitigating Potential Harm of Using Armed Security in Project Areas

Although small, the number of projects that intend to use security personnel is rising, but very few of those describe how they will mitigate potential harm on or near project sites. According to the World Bank’s Environmental and Social Framework, “the presence of security personnel (employees of a private security company, police or military personnel) can pose risks to, and have unintended impacts on, both project workers and local communities”(World Bank 2018e). For example, the way in which security personnel interact with communities and project workers may appear threatening to them or may lead to conflict. As of December 2020, although only 4 percent of investment projects (29 out of 823) in conflict-affected countries indicate the intended use of security personnel, with one exception, all of these were approved after FY15 (and one-third in FY20). Of these 29 projects, only one-third refer to mitigation measures, such as memorandums of understanding, use of a complaint or grievance redress mechanism, specific oversight and accountability procedures, or security risk assessments. Projects that include mitigation measures are approved almost exclusively after the provision of enhanced guidance by the Bank Group, rolled out in investment projects as of October FY19. In addition, task teams have been supported by a recent Technical Note, The Use of Military Forces to Assist COVID-19 Operations, which includes suggestions on how to mitigate risk. The revised OP2.30, “Development Cooperation and Conflict,” includes guidance on the exceptional use of security and military agencies in project implementation in emergencies and insecure areas.