This blog is part of the series Learning from Fragility that aims to inform discussions at the Fragility Forum by exploring lessons from past support for development in fragile and conflict settings.

World Bank Development Policy Financing (DPF) is the provision of budget support in return for which a government takes specific reform actions (referred to as “prior actions”). DPF is an increasingly important tool for the Bank to assist countries affected by fragility, conflict, or violence (FCV). While the number of DPF operations in FCV settings has averaged around 10 per year in the last decade, average commitment size has increased sharply, having tripled from US$43 million over Fiscal Year (FY) 10-15 to US$127 million over FY16-22 (Figure 1). 

To a significant extent, this rapid increase in the value of DPF commitment reflects a few particularly large operations, some of over US$1 billion each. FCV DPF to the top five receiving countries (Iraq, Sudan, Cote d’Ivoire, Afghanistan, and Niger – of which all but Cote d’Ivoire remain on the World Bank’s list of fragile and conflict-affected situations in FY22) totaled US$6.6 billion in this period, just over half of total Bank budget support to countries in fragile situations. That said, of the 53 countries on the World Bank’s FCV list, 35 have had at least one development policy operation (DPO) in the last decade.

Figure 1. World Bank DPF in Fragile and Conflict-affected Situations

 


How does the performance of DPOs in fragile or conflict-affected contexts compare with other DPOs?

In most respects, DPOs in FCV achieved better Outcome ratings when compared with DPOs over the last decade (Figure 2, left side). While about three-quarters of DPOs (both FCV and the broader universe of DPOs) were rated Moderately Satisfactory or higher (MS+) for Development Outcome, DPOs in FCV contexts were rated Satisfactory almost twice as often. At the same time, outcome ratings for DPOs more generally were dominated by Moderately Satisfactory ratings. This is an encouraging result in terms of the effectiveness of Bank DPF in FCV situations – better outcomes are, after all, better. 

 

Figure 2: Outcome and Bank Performance Ratings:  All DPOs vs. FCV DPO ratings

 

While Outcome ratings measure progress in reaching results indicator targets due to the policies supported by the DPO, outcomes can be significantly influenced by exogenous factors or shocks. Therefore, failure to meet targets is not always the result of poorly designed or executed operations. 

Ratings are rubrics for assessing performance relative to a project or program’s objectives. For Development Policy Operations, the “outcome” rating brings together two dimensions: relevance (of prior actions) and efficacy (achievement of objectives).  IEG also rates the relevance of results indicators and Bank Performance.

This is one of the reasons the World Bank and IEG also rate Bank Performance. These ratings are intended to assess those aspects of DPO design and implementation that are largely within the control of the World Bank. Historically, Bank Performance ratings are derived from several criteria: “Quality at Entry” (a measure of the quality of DPO design); quality of monitoring and evaluation; and quality of supervision (although these criteria changed in 2020 – see below). Figure 2 (right side) compares ratings for Bank Performance of DPOs in FCV versus non-FCV contexts. It suggests DPOs in FCV contexts perform worse than in non-FCV contexts, with 38% of operations in FCV contexts achieving Bank Performance ratings of Moderately Unsatisfactory or lower compared to 19% for all DPOs. Interestingly, Bank Performance for DPOs in FCV has a more polarized distribution than do the universe of DPOs, with about half of FCV DPOs having a rating of Satisfactory, compared with about a quarter of all DPOs.

Before drawing any conclusions about the relative performance of DPOs in FCV versus non-FCV contexts, it is worth looking at performance over time during the last decade. Figures 3 and 4 compare Outcome and Bank Performance ratings between the first and second halves of the decade. For the universe of DPOs, MS+ outcome ratings have improved somewhat between the period (from 75% to 80% of operations). In comparison, those for FCV DPOs have worsened markedly (from just 73% to 55%), indicating a reversal of the finding that DPOs in FCV contexts perform better than do non-FCV DPOs. 

Figure 3: All Development Policy Operations Ratings FY10-15 vs. FY16-20

Figure 4: Development Policy Operations in FCV Ratings FY10-15 vs. FY16-20

The trend in ratings for Bank Performance is similar, with the share of MS+ Bank Performance ratings for all DPOs having risen from 73% to 83% while that for DPOs in FCV contexts fell from 88% to 78%. Whether or not this finding derives from more complex and challenging FCV situations in the latter half of the decade, or a deterioration in program design or implementation, is beyond the scope of this blog. One part of the story could be the nature of the reforms supported by DPF in FCV (as measured by prior actions). While most prior actions in FCV DPOs remained focused on reforms at the level of the Central Government, this share was 14% lower in the second half of the decade than during the first half. But a more granular assessment of prior actions would be needed before making too much of this change.

Rethinking what success looks like in fragile contexts

Perhaps the explanation lies elsewhere. Over the last two years, the Bank and IEG adjusted how they assess Bank Performance in DPOs to better reflect differences with more traditional investment projects. Recognizing that DPO design and implementation is as much an art as a science, the Bank and IEG piloted an approach that more explicitly considered the range of factors that experience suggested were important contributors to success. These include the adequacy of the analytical underpinnings guiding the selection of reforms supported (and associated prior actions), the quality of the ex-ante assessment of risks that the operation would not achieve its development objectives, the extent to which mitigating measures were planned to reduce these risks, and evidence of learning from experience in program design.  

More recently, in response to a request from IDA Deputies, representatives from the now 50+ donor governments who meet every three years to replenish resources and review the policy framework of the Bank Group’s fund for the poorest countries, additional guidance for assessing Bank Performance with DPOs for FCV situations was adopted. This was intended to more explicitly recognize that fragile situations are inherently volatile and uncertain and that outcomes are highly likely to be impacted by exogenous or unanticipated factors. As such, the piloted changes to evaluate Bank Performance introduced in 2020, which were seen as particularly relevant to an FCV context, were formalized. Greater attention was also given to anticipating potential project impacts on fragility and conflict drivers, institution building, and to coordination and consultation with a broader range of development partners, potentially including UN agencies and diplomatic and security actors.  

To date, 10 DPOs in FCV contexts have been evaluated by IEG using the new framework. All 10 have had Bank Performance rated Moderately Satisfactory or better. This is encouraging news and is evidence that the Bank is building its capacity to engage and navigate the policy side of development support in complex situations. It also suggests a need to have a more nuanced appreciation of what “good” looks like in volatile and uncertain situations, where success is incentivized not through risk avoidance but through “informed risk-taking.

For more details on World Bank Group ratings, please visit the IEG Data page. 


Photo: The “Saving Iraqi Culture” monument in Baghdad designed by sculptor Mohammed Ghani Hikmat. Iraq is one of top five DPF recipients in FCV settings. Credit: Rasool Ali/Shutterstock