Early-Stage Evaluation of the Multiphase Programmatic Approach
Overview
The World Bank introduced the multiphase programmatic approach (MPA) in 2017 as a means of structuring a long, large, or complex engagement as a set of shorter linked projects or phases using either investment project financing or Program-for-Results financing.1 This engagement was intended to take either a vertical form within a single country, typically over 8–10 years, or a horizontal form across several countries at the same time (or several states within a country), supporting activities that were either scalable or modular or that followed a predictable course, provided that in each case they were consistent with the program development objective.
The motivation behind the MPA, outlined in a 2017 paper from the Board of Executive Directors, was to provide continuity of engagement, allow more flexibility in responses to changed circumstances, encourage adaptive learning, and support stepwise progress toward a long-term development objective (World Bank 2017). By signaling a willingness to pursue a long-term development objective, the MPA was intended to strengthen the coherence of World Bank–financed interventions, contribute to building consensus on the client side, and diminish the likelihood of interruptions in support between phases. Compared with a single large operation, the MPA would enable more adaptation to circumstances by incorporating multiple opportunities for reflection and course corrections.
The 2017 Board paper also expected benefits to accrue operationally. By committing financing in smaller phases, rather than committing the total cost of the larger program up front, the World Bank could reduce undisbursed balances and allow borrowers to save on commitment fees, as well as reduce the processing costs of follow-up phases relative to stand-alone operations. The 2017 Board paper also noted that the MPA might provide a framework for engagement by other lenders even beyond the duration of World Bank financing. For instance, the World Bank could mobilize commercial financing for the first phase of an infrastructure program or could partner with other multilateral development banks, commercial lenders, private investors, or private companies to finance or implement subsequent phases.
Since the introduction of the MPA, there has been a steady increase in its use. At the end of December 2023, the portfolio comprised 11 horizontal and 29 vertical MPAs. Most nonemergency MPA financing occurs in International Development Association–eligible countries and targets infrastructure and sustainable development, with agriculture, energy, water, and transport being the most important users. MPAs have been used in all World Bank Regions. Three-quarters of nonemergency MPA financing has been for the Africa Region, 85 percent has been for International Development Association–eligible countries, and 20 percent has been for fragile and conflict-affected countries.
This evaluation assesses the performance of the approach against the expectations outlined in the 2017 Board paper. The youth of the MPA portfolio means that no ex post assessment of outcomes is possible. The evaluation instead asks if the design of MPAs is fulfilling the expectations outlined in the 2017 Board paper, if the specific features of MPAs are functioning as expected, and if there is an enabling environment for MPAs within the World Bank and on the client side. The evaluation also assesses the claims made in the Operations Policy and Country Services technical briefing to the Board on the processing times for MPAs relative to non-MPAs. It does not assess the uptake of MPAs, assess the suitability of the instrument where it has not been applied, ask if the policy scope of MPAs or the delegation of authority for MPAs should be expanded, or assess the improved efficiency of management of the World Bank Group’s financial resources.
Specifically, the evaluation assesses whether the design and early implementation of the MPA has supported the objectives on which the approach’s effectiveness depends. These objectives are as follows:
- Coherence. A coherent program fits within the broader program at the level of the country, sector, and institution. The MPA is expected to be more coherent than its alternatives because it was intended to leverage external partnerships and internal collaboration more effectively.
- Continuity. This refers to the MPA’s ability to provide stable, long-term support. The vertical MPA supports continuity better than its alternatives because of its programmatic structure and the provision for overlapping phases.
- Learning. Although all operations should embed knowledge, the MPA requires an explicit learning plan, with specificity on implementation arrangements and how the knowledge is to be used.
- Adaptation. This refers to the ability of the MPA to adjust the content and timing of its phases in response to new information, evolving priorities, and changing context due to having a larger number of preset points for stocktaking than would be present in a single operation.
Main Findings
Overall, the evaluation finds that so far, the MPA meets expectations on learning and continuity, although it is less of a departure from business as usual than was hoped for on coherence and adaptation. There are positive indications of support for learning and continuity from the MPA, with no observable differences so far on coherence or adaptation. The ex ante objectives of MPAs are not set at a higher outcome level than comparators, nor are they more likely to measure institutional strengthening,2 which may be useful in more challenging fragility, conflict, and violence–affected environments. MPAs better support and measure climate-related objectives. With respect to expectations from the MPA, the evaluation finds the following:
- Coherence. There is no evidence yet that MPAs are more tightly anchored in Country Partnership Frameworks than other engagements. Respondents view the longer-term horizon of vertical MPAs and the regional aspect of horizontal MPAs as more effective in motivating partnerships with other donors, but this is not yet reflected in cofinancing data. Neither vertical nor horizontal MPAs appear to support collaboration within the World Bank any better than the alternatives.
- Continuity. All those vertical MPAs that have moved beyond a first phase have done so without a break in support. Both vertical and horizontal MPAs provide flexibility in the timing of additional phases that can be tailored to country circumstances. Although risks to continuity have not materialized significantly, there is a perception that MPAs better manage these risks. Only two MPAs have been converted to stand-alone projects over the evaluation period.
- Learning. Most MPAs have adequate learning agendas. There is also evidence that lessons learned have informed follow-up phases of both horizontal and vertical MPAs and that learning is perceived as more effective under MPAs. At the same time, there is no discernible difference from other operations in how learning is financed (mainly through trust funds), and reporting on the implementation of learning plans is mainly on the preparation of follow-up phases.
- Adaptation. Given that the approach is relatively new, there is little evidence that the frequency of, motivations for, and content of restructuring under MPAs are any different from non-MPAs. However, for the small number of MPAs that have progressed beyond phase 1, there is evidence that learning is informing the design of subsequent phases.
The growing use of MPAs highlights the need to consider the trade-offs among scale, speed, and complexity. The expectation for MPAs to deliver at scale and with speed is linked to the use of emergency response MPAs during the COVID-19 Strategic Preparedness and Response Plan. These emergency MPAs benefited from key fiduciary and operational flexibilities, which significantly contributed to their success, allowing for rapid disbursement and the swift achievement of project development objectives. Although MPAs can still deliver with scale and speed, achieving these objectives will require that the project design and implementation features are intentionally geared toward these objectives. Increased complexity could slow down implementation when speed and replicability are explicit objectives.
Implications
The evaluation finds several areas for management to consider as the World Bank increases its use of the MPA that could strengthen the effectiveness of the approach. However, it notes that the portfolio is for the most part at an early stage of implementation, and all programs are still active, with only 7 out of the 40 having moved beyond phase 1 during the evaluation period. The benefits and risks of using the approach, particularly the absence of a firm commitment on either side to follow-up phases, should be better explained to clients at the outset. Among the issues for consideration are the following:
- Outcomes. It is important to ensure that MPA objectives and their targets, including outcomes and contributions to high-level outcomes, are set at an appropriate level of ambition and adjusted to reflect changing circumstances over the lifetime of the MPA and that targets are ratcheted up where possible.
- Coherence. The justification for using either type of MPA should be clearly articulated in Country Partnership Frameworks. This would include clarity on how the MPA fits into the larger World Bank Group program and, where relevant, how the program complements interventions by partners. Political consensus around a long-term objective or strategy can facilitate the implementation of vertical MPAs, while regional organizations, platforms, or specific mechanisms designed to intermediate knowledge across countries can similarly support the horizontal MPAs.
- Continuity. The prioritization of MPAs in Country Partnership Frameworks should be considered, particularly for International Development Association–eligible countries subject to greater uncertainty over funding allocations, where funding trade-offs may be most acute and shocks more prevalent. The leveraging of MPAs through alternative sources of external finance (for example, climate finance or debt swaps) should also be considered. There is a strong need for dedicated training and networking opportunities to ensure that team leaders are well equipped with operational, technical, and interpersonal skills, as well as sector and country knowledge, to manage the complexities of implementing MPAs, especially those requiring coordination across sectors and countries.
- Learning. MPA learning activities need to be properly tailored to project activities, adequately resourced and monitored, and able to adapt to incorporate “learning moments” to strengthen feedback. Learning should also encompass institutional development over the program cycle. This learning may entail developing indicators that, for vertical MPAs, measure the effectiveness of long-term institutional reforms and, for horizontal MPAs, incentivize and measure the effectiveness of collaboration among participants.
- Each phase in the multiphase programmatic approach follows the policies and procedures for the lending instrument it uses, including those for restructuring, though approval of additional financing within the program envelope is delegated to management.
- Findings on outcome orientation and institutional development were confirmed using t tests. Chi-squared tests of outcomes capturing institutional development suggest that comparators outperform multiphase programmatic approaches. This is mainly because Operations Policy and Country Services guidance is for multiphase programmatic approaches to have program development objective indicators at the beneficiary level.