Main Findings

Overall, the MPA has partly met the expectations set at its creation. This evaluation cannot assess the MPA’s relative performance in helping achieve long-term outcomes because of the youth of the MPA portfolio, but the findings currently show that although there are positive indications of support for learning and continuity, so far there is little difference on coherence and adaptation. In addition, the ex ante objectives of MPAs are not set at higher outcome levels than those of comparator projects and programs, nor are MPAs more likely to support or measure institutional strengthening, which may be critical in more challenging environments that are experiencing FCS. We also found the following with respect to the specific design expectations from the MPA:

1. Coherence. There is no evidence yet that MPAs are more tightly anchored in CPFs than other engagements. Respondents view the longer-term horizon of vertical MPAs and the regional aspect of horizontal MPAs as more effective in motivating partnerships with other donors, but this is not yet reflected in cofinancing data. Neither vertical nor horizontal MPAs appear to support collaboration within the World Bank any better than the alternatives.
2. Continuity. All those vertical MPAs that have moved beyond a first phase have done so without a break in support. Both vertical and horizontal MPAs provide flexibility in the timing of additional phases that can be tailored to country circumstances. Although risks to continuity have not materialized significantly, there is a perception that MPAs better manage these risks. Only two vertical MPAs are being converted to stand-alone engagements.
3. Learning. Most MPAs have adequate learning agendas. There is also evidence that lessons learned have informed follow-up phases of both horizontal and vertical MPAs and that learning is perceived as more effective under MPAs. However, there is no discernible difference from other operations in how learning is financed (mainly through trust funds), and reporting on the implementation of learning plans is mainly reflected in the preparation of follow-up phases.
4. Adaptation. Given that the approach is relatively new, there is little evidence that the frequency of, motivations for, and content of restructuring under MPAs are any different from non-MPAs. However, for the small number of MPAs that have progressed beyond phase 1, there is evidence that learning is informing the design of subsequent phases.

Implications

This evaluation has highlighted several areas in which MPAs’ effectiveness might be strengthened, while noting that the portfolio is for the most part at an early stage of implementation and all programs are still active, with only 7 out of the 40 having moved beyond phase 1 during the evaluation period. Although MPAs can still deliver with scale and speed, achieving these objectives will require that the project design and implementation features are intentionally geared toward these objectives. The benefits and risks of using the approach, particularly the absence of a firm commitment on either side to follow-up phases, should be better explained to clients at the outset. Other issues for consideration are as follows:

• Outcomes. It is important to ensure that MPA objectives and their targets, including outcomes and contributions to high-level outcomes, are set at an appropriate level of ambition; that they are adjusted to reflect changing circumstances over the lifetime of the MPA; and that targets are ratcheted up where possible.
• Coherence. The justification for using either type of MPA should be clearly articulated in CPFs. This would include clarity on how the MPA fits into the larger World Bank Group program and, where relevant, how the program complements interventions by partners.
• Continuity. The prioritization of MPAs in CPFs should be considered, particularly for IDA-eligible countries subject to greater uncertainty over funding allocations, where funding trade-offs may be most acute and shocks more prevalent. The leveraging of MPAs through alternative sources of external finance (for example, climate finance or debt swaps) should also be considered. Attention should be given to providing dedicated training and networking opportunities to MPA TTLs, including local staff. It is critical to ensure team leaders are well equipped with operational, technical, and interpersonal skills, as well as sector and country knowledge, to manage the complexities of implementing MPAs, especially those requiring coordination across sectors and countries. Transitions across TTLs need to be especially carefully managed.
• Learning. MPA learning activities need to be properly tailored to project activities, adequately resourced and monitored, and able to adapt to incorporate “learning moments” to strengthen feedback. Learning should also encompass institutional development over the program cycle. This learning may entail developing indicators that, for vertical MPAs, measure the effectiveness of long-term institutional reforms and, for horizontal MPAs, incentivize and measure the effectiveness of collaboration among participants.

The growing use of MPAs highlights the need for consideration of the trade-offs among scale, speed, and complexity. The expectation for MPAs to deliver at scale and with speed is linked to the use of emergency response MPAs during the COVID-19 Strategic Preparedness and Response Plan. These emergency MPAs benefited from key fiduciary and operational flexibilities, which significantly contributed to their success, allowing for rapid disbursement and the swift achievement of PDOs. Although MPAs can still deliver with scale and speed, achieving these objectives will require that the project design and implementation features are intentionally geared toward these objectives. Increased complexity could slow down implementation when speed and replicability are explicit objectives.