Results and Performance of the World Bank Group 2024
Chapter 4 | Multilateral Investment Guarantee Agency
Highlights
The Multilateral Investment Guarantee Agency (MIGA) is behind on submitting self-evaluations of its guarantee projects, which casts doubt on the validity of its development outcome ratings. Given the available data, development outcomes of MIGA guarantee projects declined slightly over the long term—from 69 percent of projects rated satisfactory or better in FY13–18 to 68 percent of projects rated satisfactory or better in FY18–23—as a result of a drop in ratings in challenging contexts and in infrastructure sector projects.
MIGA achieves foreign investment–level outcomes less often than project-level outcomes. This is a concern because MIGA’s business model involves guaranteeing foreign direct investments, which implies that it should achieve foreign investment–level outcomes at least as often as project-level outcomes.
The new World Bank Group guarantee platform provides MIGA with the potential to enhance its supervision beyond environmental and social assessment and country risk assessment, but it also poses risks given MIGA’s limited experience in monitoring project results during supervision.
RAP’s main source of evidence for MIGA guarantee projects is MIGA’s annual self-evaluation of almost all of its guarantee projects; IEG evaluates those that were canceled. For each MIGA guarantee project that has reached early operating maturity (so that sufficient information is available for the evaluation), MIGA underwriting staff conduct a self-evaluation that IEG independently validates. IEG evaluates canceled MIGA guarantee projects in lieu of MIGA self-evaluations. In this section, when we refer to projects “evaluated and validated by IEG,” we mean this set of projects.
Figure 4.1. Performance Ratings in Multilateral Investment Guarantee Agency Guarantee Projects
Figure 4.1. Performance Ratings in Multilateral Investment Guarantee Agency Guarantee Projects
Source: Independent Evaluation Group.
Note: MIGA = Multilateral Investment Guarantee Agency.
MIGA guarantee projects’ performance is assessed on three dimensions: development outcome, MIGA role and contribution, and MIGA work quality. Figure 4.1 shows these dimensions and their respective indicators. Development outcome is particularly important for this section and measures performance across four indicators: project business performance, economic sustainability, environmental and social effects, and foreign investment effects. Development outcomes are rated on a six-point scale: highly successful, successful, mostly successful, mostly unsuccessful, unsuccessful, and highly unsuccessful. Before FY20, the ratings were based on a four-point scale: excellent, satisfactory, partly unsatisfactory, and unsatisfactory.
Trends
Development outcomes of MIGA guarantee projects declined slightly over the long term as a result of a drop in ratings in challenging contexts and in infrastructure sector projects. For MIGA guarantee projects, we analyzed the trends in development outcomes over a six-year rolling average because of the small number of projects evaluated annually. For this analysis, the long term refers to the period between FY13–18 and FY18–23. In this subsection, we report a slight long-term decline in MIGA’s overall development outcomes. MIGA classifies its guarantee projects into four sectors: Agribusiness and General Services, Energy and Extractive Industries, Finance and Capital Markets, and Infrastructure. We found that Infrastructure sector projects, particularly those in IDA and blend countries and in Sub-Saharan Africa, contributed negatively to MIGA’s overall development outcomes. However, these projects are in challenging contexts.
Development outcomes of MIGA guarantee projects declined slightly over the long term. Development outcome ratings of MIGA guarantee projects declined from 69 percent of projects rated satisfactory or better in FY13–18 to 68 percent in FY18–23 (figure 4.2). In the remainder of this subsection, we analyze the development outcomes for two subgroups: (i) IDA and blend countries and (ii) Sub-Saharan Africa. These subgroups contributed negatively to MIGA’s overall development outcomes. Projects in the Infrastructure sector in these subgroups also contributed negatively to their development outcome ratings.
Projects in IDA and blend countries contributed negatively to MIGA’s overall development outcomes. Development outcomes of projects in IDA and blend countries declined from 74 percent of projects rated satisfactory or better to 50 percent over the long term. However, if ratings of projects in IDA and blend countries were not considered, then the percentage of MIGA projects overall with development outcome ratings of satisfactory or better would have increased by 14 percentage points. This negative contribution to MIGA’s overall development outcome rating is expected because IDA and blend countries are challenging contexts.1
Figure 4.2. Development Outcomes for Multilateral Investment Guarantee Agency Guarantee Projects, FY13–23
Figure 4.2. Development Outcomes for Multilateral Investment Guarantee Agency Guarantee Projects, FY13–23
Source: Independent Evaluation Group, Multilateral Investment Guarantee Agency database.
Note: The Multilateral Investment Guarantee Agency Project Evaluation Report guidelines were changed in FY19, replacing a four-point scale for development outcome ratings with a six-point scale. The six-point rating scale, applied to projects starting in FY20, was converted to a four-point one as follows: highly successful = excellent; successful and mostly successful = satisfactory; mostly unsuccessful = partly unsatisfactory; and highly unsuccessful and unsuccessful = unsatisfactory. PER = Project Evaluation Report.
Similarly, projects in Sub-Saharan Africa contributed negatively to MIGA’s overall development outcomes. Development outcomes of projects in Sub-Saharan Africa declined from 72 percent of projects rated satisfactory or better to 50 percent over the long term. However, if ratings of projects in Sub-Saharan Africa were not considered, then the percentage of MIGA projects overall with development outcome ratings of satisfactory or better would have increased by 9 percentage points. Sub-Saharan Africa is a challenging context, so this negative contribution to MIGA’s overall development outcomes is expected.
Figure 4.3. Development Outcomes for Multilateral Investment Guarantee Agency Guarantee Projects by Region, FY13–18 Versus FY18–23
Figure 4.3. Development Outcomes for Multilateral Investment Guarantee Agency Guarantee Projects by Region, FY13–18 Versus FY18–23
Source: Independent Evaluation Group, MIGA database.
Note: EAP = East Asia and Pacific; ECA = Europe and Central Asia; LAC = Latin America and the Caribbean; MENA = Middle East and North Africa; MIGA = Multilateral Investment Guarantee Agency; S+ = satisfactory or better; SAR = South Asia; SSA = Sub-Saharan Africa.
Infrastructure sector projects are a secondary factor in the rating decline in challenging contexts such as IDA and blend countries and Sub-Saharan Africa. Examples of Infrastructure sector projects in challenging contexts include the construction of a dam and the construction of a megawatt power plant in IDA countries in Sub-Saharan Africa, both of which were rated mostly unsuccessful or worse on development outcomes. Development outcomes of projects in the Infrastructure sector declined from 74 percent of projects rated satisfactory or better to 63 percent over the long term (figure 4.3). In addition, a substantial share of Infrastructure sector projects evaluated and validated by IEG in FY18–23 was in IDA and blend countries (52 percent) and in Sub-Saharan Africa (26 percent). However, if ratings of Infrastructure sector projects were not considered, then the percentage of MIGA projects in IDA and blend countries with development outcome ratings of satisfactory or better would have increased by 16 percentage points, and the percentage of MIGA projects in Sub-Saharan Africa with satisfactory or better ratings would have increased by 11 percentage points. These negative contributions to IDA and blend and Sub-Saharan Africa ratings overall are expected because of the challenging contexts and inherent complexities of Infrastructure sector projects.
Challenges
Critical challenges for MIGA include cost overruns and construction delays, project company quality, legal and regulatory risk, and achieving foreign investment–level outcomes. In this subsection, we identify the most prevalent factors linked to development outcomes: cost overruns and construction delays, project company quality, and legal and regulatory risk. (MIGA’s direct client is its guarantee holder, which is the company benefiting directly from the MIGA guarantee. The guarantee holder has direct influence over a project company, which owns and implements the project. Project company quality refers to the ability, technical expertise, and track record of the project company.) We focus on factors that are not directly within MIGA’s influence: cost overruns and construction delays, project company quality, and legal and regulatory risk. Finally, we show that MIGA guarantee projects achieve foreign investment–level outcomes less often than they achieve project-level outcomes.
The most prevalent factors linked to MIGA’s development outcomes are related to cost overruns and construction delays, project company quality, and legal and regulatory risk. RAP 2024 conducted a deep dive of 26 MIGA guarantee projects evaluated and validated by IEG (FY20–23) using the RAP 2023 taxonomy to identify the top factors linked to development outcomes. A factor can have both positive and negative links to development outcomes (figure 4.4). The most prevalent factor is cost overruns and construction delays, which occurred in 46 percent of projects reviewed. Project company quality occurred in 31 percent of the projects. Finally, legal and regulatory risk occurred in 27 percent of the projects. None of these factors are under MIGA’s direct control, as discussed in this subsection. However, MIGA can indirectly influence cost overruns, construction delays, and project company quality through its guarantee holder. This is only true during the underwriting stage because MIGA limits its supervision to environmental and social assessment and country risk assessment.
Figure 4.4. Most Prevalent Factors Linked to Development Outcomes
Figure 4.4. Most Prevalent Factors Linked to Development Outcomes
Source: Independent Evaluation Group, Multilateral Investment Guarantee Agency database.
Note: “Positive direction” means the extent to which the factor contributes positively to the development outcomes of projects evaluated and validated by the Independent Evaluation Group. “Negative direction” means the extent to which the factor contributes negatively to the development outcomes of projects evaluated and validated by the Independent Evaluation Group.
MIGA can indirectly influence project company quality by selecting the right guarantee holder. Project company quality is defined as the quality and experience of the management team implementing the project and their technical skills, track record, contractor competency, familiarity, and acumen. MIGA uses factors such as management experience, sector experience, and commitment to select a guarantee holder, which, in turn, selects a project company to implement a project financed by the guarantee holder.2 Project company quality has a positive influence on MIGA’s development outcomes in 75 percent of projects. For example, the selection of a guarantee holder with deep experience in the health-care sector that selected a strong project company led to the earlier-than-expected completion of a hospital construction project in Central Asia and Türkiye at a lower budget than expected.
Cost overruns and construction delays are generally outside of MIGA’s direct influence. This is a negative factor in two-thirds of the projects. Because MIGA’s supervision is limited, it does not directly influence cost overruns or construction delays. However, guarantee holders with strong qualities tend to select project companies that prevent cost overruns and construction delays. Therefore, MIGA must carefully select the right guarantee holder during the underwriting stage to help prevent cost overruns and construction delays. For example, a large-scale power plant construction project in the Middle East was delayed by three years because MIGA’s guarantee holder selected an unsuitable project company. Early diligence on selection of the guarantee holder is important because MIGA limits its supervision to environmental and social assessment and country risk assessment.
Legal and regulatory risk is outside of MIGA’s influence. Legal and regulatory risk is defined as risk related to regulatory policies, government, legislation, and bureaucratic mechanisms. This factor has a positive influence on development outcomes in about 85 percent of MIGA projects. However, it has a negative influence in about 15 percent of projects. For example, financial performance of a bulk water treatment facility in Sub-Saharan Africa was negatively affected when the government refused to renegotiate a tariff after an unexpected increase in the project’s construction costs. Given MIGA’s role as a guarantor, it cannot influence governments to change laws, regulations, policies, or bureaucratic mechanisms.
Foreign investment–level outcomes are achieved less often than project-level outcomes for MIGA guarantee projects, which is a concern given MIGA’s business model. RAP 2024 conducted a deep-dive analysis of 15 MIGA guarantee projects evaluated and validated by IEG during FY21–23 to identify project-level and foreign investment–level outcomes and their achievement rates. Eighty-one percent of the 58 project-level outcomes were achieved (fully or partially), whereas 66 percent of the 18 foreign investment–level outcomes were achieved (figure 4.5).3 Given that MIGA’s primary purpose is to support foreign direct investment, one would expect that it would be at least as good at achieving foreign investment–level outcomes as it is at achieving project-level outcomes. (MIGA launched IMPACT [Impact Measurement and Project Assessment Comparison Tool], an ex ante impact measurement tool, in FY20. Although both IMPACT and AIMM, IFC’s ex ante monitoring tool, track project outcomes, IMPACT tracks foreign investment, whereas AIMM tracks market creation. This reflects MIGA’s more focused mandate to promote development through foreign investment.)
Figure 4.5. Project-Level Versus Foreign Investment–Level Outcome Achievement Rates
Figure 4.5. Project-Level Versus Foreign Investment–Level Outcome Achievement Rates
Source: Independent Evaluation Group, Multilateral Investment Guarantee Agency database.
Levers
MIGA could learn more about development outcomes and factors linked to them by promptly delivering self-evaluations and enhancing supervision. In this subsection, we argue that MIGA could catch up on its self-evaluations, which are significantly backlogged, to accurately measure its development outcomes. We also point out that MIGA has the potential to go beyond environmental and social assessment and country risk assessment and monitor project results in its supervision as it takes ownership of the new Bank Group guarantee platform.
MIGA is behind on submitting self-evaluations of its guarantee projects, which prevents us from having an accurate picture of its overall development outcome ratings. All MIGA guarantee projects are subject to self-evaluation. IEG reviews all completed self-evaluations and validates them to assess the ratings independently. IEG evaluations of MIGA projects enable IEG and MIGA to account to the Board for achievements and contribute to learning. However, 19 MIGA projects are pending self-evaluations during the FY21–23 period (45 percent of planned self-evaluations during this period). Of these 19 projects, 11 involved engagements from MIGA’s legal team, including renegotiation of possible investment term modifications. As a result, self-evaluations of these MIGA projects were postponed.4 The corrective actions and pending self-evaluations from MIGA could influence its development outcome ratings. This, in turn, could affect MIGA’s reporting for the new Scorecard.
The new Bank Group guarantee platform provides MIGA with the potential to enhance its supervision beyond environmental and social assessment and country risk assessment, but it also poses risks given MIGA’s limited experience in monitoring project results during supervision. Starting July 1, 2024, the Bank Group guarantee platform, housed at MIGA, has brought together guarantee products from the World Bank, IFC, and MIGA. It will serve as a one-stop shop for all Bank Group guarantees. This platform aims to boost Bank Group annual guarantee issuance from $10.3 billion in 2024 to $20 billion by 2030. Given that the platform will encompass guarantees from all three Bank Group institutions, MIGA now has an opportunity to harmonize its monitoring during supervision with the World Bank and IFC. In doing so, it could go beyond environmental and social assessment and country risk assessment and thereby take a more proactive approach to supervision by monitoring project results.
- While MIGA’s performance in FCS was above the MIGA-wide average both in FY13–18 (78 percent versus 69 percent) and in FY18–23 (75 percent versus 68 percent), the results must be treated with caution because of a smaller cohort size of only 14 projects in total in FCS during FY13–18 (that is, an average of less than 2 projects per year and about 12 percent of MIGA projects evaluated and validated by IEG in FY13–23).
- MIGA’s business of providing risk coverage to project sponsors, shareholders, or financiers to project companies is different from direct lenders such as the International Bank for Reconstruction and Development, IDA, and IFC. The difference of the business model has an impact on the level of influence on project company.
- Based on the consultations between MIGA and IEG, it was agreed to revise project maturity for project evaluations from three to five years, thereby giving more time for the outcomes to be realized and observed.
- MIGA does not conduct monitoring and supervision of its guarantees, except for environmental and social issues. As a standard practice before the pandemic, a field visit was conducted as part of MIGA self-evaluation to collect necessary evidence for outcome measurement, but such exercise was not undertaken during the pandemic. The lack of field visits constrained the data gathering from the project. MIGA restarted the self-evaluation based on field visit from the third quarter of FY24 and intensified country visits to cover multiple projects planned for FY25, including projects that were postponed in the previous fiscal years.