Key Messages

This evaluation represents the first stage of the Independent Evaluation Group’s assessment of the World Bank’s performance in supporting more, better, and more inclusive jobs through International Development Association (IDA) financing. The scope of this evaluation is relatively narrow, focusing on direct IDA support to the achievement of IDA’s jobs objectives. Additional evaluations would be required to consider the broader jobs and economic transformation agenda.

The evaluation assesses the analytical underpinnings and operational relevance of the evolving IDA jobs strategy and the extent to which it has been translated into well-designed and well-performing jobs interventions that directly address the objectives of more, better, and more inclusive jobs.

The IDA jobs strategy successfully stimulated country analytics and diagnostics to fill knowledge gaps.

IDA policy commitments adapted well to reflect learning from jobs diagnostics. Most jobs diagnostics have incorporated recommendations that (i) feed into operational work and improve how Country Partnership Frameworks articulate jobs-related objectives and theories of change and (ii) help make jobs more central to engagements with countries affected by fragility, conflict, and violence.

The enhanced focus on jobs in IDA’s overall strategy has not been associated with a significant increase in the size of the jobs portfolio but has led to a change in the mix of jobs interventions.

The design of most jobs interventions was well grounded in analysis (including impact evaluation), and projects adequately combined multiple interventions.

The IDA jobs strategy prioritization of youth and women’s employment resulted in more focus on these two beneficiary groups in jobs interventions.

The use of jobs-relevant development policy financing prior actions has increased steadily, yet they remain infrequent and have had an impact only in one-third of validated operations.

However, the IDA jobs strategy’s promise of improved results measurement has yet to be fulfilled. Although impact evaluations are more frequent, results frameworks continue to track outputs rather than outcomes, and many countries’ labor market statistics are still lacking.

Overall, the performance of jobs interventions under implementation is on track. Two-thirds of jobs-related indicators in projects that have passed their Mid-Term Reviews were on track to meet their targets. However, little is known about the effectiveness of the interventions because of weaknesses in results measurement or in country-level statistical systems measuring labor market developments.

More, better, and more inclusive jobs are critical for poverty reduction and shared prosperity in countries eligible for International Development Association (IDA) financing. Since 2014, IDA has included jobs as a special theme, and subsequent IDA Replenishments have had what this evaluation terms an “IDA jobs strategy,” with explicit objectives, a series of policy commitments to achieve them, and results indicators to track them. This evaluation is the first in a potential series to address the World Bank’s contribution to the jobs agenda, and as such has a focused scope. It assesses the implementation of the IDA jobs strategy and answers two questions. The first is to what extent IDA’s strategy on jobs was grounded in sound analytics, adaptive, and operationally relevant. The second is to what extent the strategy has been translated into relevant and effective jobs interventions that directly address the objectives of more, better, and more inclusive jobs.

The evaluation is based on a conceptual framework that identifies three direct, interrelated channels through which IDA support for jobs objectives is pursued: acting on labor demand (including efforts to expand the demand of private sector firms for workers and create the conditions for reliable and adequately remunerated self-employment); increasing labor supply (including efforts to improve the marketable skills of workers, expand the labor force participation of women, and support youth employment); and improving labor market flexibility (including policies to facilitate the movement of workers from lower-productivity to higher-productivity activities within sectors, and from lower-value-added to higher-value-added sectors) and geographic mobility where lack of mobility is identified as a significant constraint in jobs diagnostics.

The evaluation focuses on IDA-supported interventions that directly support jobs objectives across the three Replenishment cycles from fiscal years 2015 to 2022. Although the structural change associated with economic transformation is linked to jobs and growth, the scope of this evaluation is limited to the three main channels for achieving IDA jobs objectives. Therefore, we do not evaluate the implementation and effectiveness of the economic transformation agenda, given (i) its very broad purview and (ii) the difficulties inherent in measuring the volume and impact of World Bank financing for economic transformation and how it is intermediated through results indicators at the project or Country Partnership Framework levels. The contribution through the IDA Private Sector Window is also outside the scope of this evaluation, which is being presented alongside the evaluation of the Private Sector Window by the Independent Evaluation Group.

International Development Association’s Evolving Jobs Strategy

The evaluation finds that, through the deployment of jobs diagnostics, the 17th Replenishment of IDA (IDA17) and IDA18 policy commitments helped address significant knowledge gaps. Previous country analytics did not focus on jobs, with little attention to the factors that influence labor market outcomes in IDA countries. Most jobs diagnostics provided actionable recommendations. In two-thirds of cases, these were tailored to country contexts and offered guidance on prioritization, even if very few distinguished between short-term and long-term interventions.

There was also a robust feedback loop between evidence from jobs diagnostics and subsequent IDA cycles. Policy commitments evolved over time to reflect learning from country evidence and experience—for example, with respect to the importance of complementing interventions aimed at raising productivity in agriculture (to improve earnings) with interventions that promote agribusiness (to create new jobs) in IDA18.

The IDA jobs strategy and jobs diagnostics contributed to a better articulation of jobs-related objectives, theories of change, and results frameworks in Country Partnership Frameworks. Whereas previously support for jobs had been implicit, typically occurring through business environment reforms, access to finance, or improvement in agricultural productivity, the strategy nudged Country Partnership Frameworks toward a more explicit and coherent statement of the jobs agenda.

The sharper focus on jobs in country strategies is associated with a slight increase in the relative size of the portfolio. The estimated average annual shares of the IDA jobs-related portfolio in the total project count and value of commitments of all IDA projects increased slightly over the evaluation period.

There was also a pronounced effect on engagements with countries affected by fragility, conflict, and violence (FCV). IDA17 policy commitments under the FCV pillar led to the development of a specific integrated jobs strategy for countries affected by FCV. As a result, the jobs portfolio in countries affected by FCV put greater emphasis on youth employment and the economic inclusion of displaced people.

However, the IDA jobs strategy’s promises of improved results measurement have not yet been fulfilled. Although impact evaluations are more frequent, results frameworks continue to track outputs rather than outcomes, and most countries’ labor market statistics are still lacking. There is an inherent tension between the IDA jobs strategy’s ambition of better capturing jobs outcomes and the systems and incentives underlying results frameworks. Although IDA Results Measurement System Tier 2 indicators capture intermediate outcomes from IDA financing,1 at the project level, teams may be discouraged from including appropriate outcome-level indicators in results frameworks, partly over concerns about attribution. Drawing on the experience with improving the focus on gender in operations, IDA Results Measurement System Tier 3 indicators could also better capture how well IDA operations articulate and track contributions to jobs outcomes. World Bank analysis has also highlighted some of the technical difficulties in estimating the indirect impact of IDA’s interventions on jobs created, which is complex and data intensive.2

Jobs Interventions

IDA commitments provoked a change in the mix of jobs interventions in the portfolio. First, consistent with the IDA19 commitment to help remove bottlenecks in sectors with high potential for private sector–led job creation, there was an increase in projects addressing labor demand in formal firms, in some cases accompanied by an explicit focus on economic diversification through support for specific value chains. Second, there was a shift toward support for agribusiness and agricultural value chains, consistent with IDA’s priority of growing jobs in the food system. Third, there was an increase in the proportion of jobs-relevant projects that sought the participation of youth and women and of projects specifically seeking to increase youth employment. Conversely, interventions that specifically sought to improve women’s employment remained rare in the portfolio. That said, reinforcing commitments made under the gender strategy and IDA policy commitments resulted in wider use of gender-disaggregated indicators in jobs-focused projects.

Development policy financing was used in several countries to contribute to jobs outcomes through changes in the labor market regulations. In the small sample of development policy financing Implementation Completion and Results Report Reviews, the Independent Evaluation Group deemed jobs-related prior actions to be relevant, but only one-third of operations achieved targets for results indicators, primarily because of a mismatch between the ambition of the reform supported and the institutional capacity for implementation.

The design of jobs interventions was well informed by evidence. The evaluation triangulated information from the portfolio review, the structured literature review, Independent Evaluation Group–validated project self-evaluations, and case studies to assess the strength of the analytical underpinning of jobs-related interventions and the quality of their design. Most jobs interventions supported by IDA were grounded in a strong evidence base. There was also evidence of learning and adaptation of operational design within and across countries. Youth employment and some productive inclusion projects, for example, showed a strong trend toward the bundling of supply-side interventions, marking an improvement since the Independent Evaluation Group’s 2012 evaluation.

However, fully integrating supply- and demand-side interventions—as recommended by many jobs diagnostics—has proven difficult. Although collaboration between the Agriculture and Food and the Finance, Competitiveness, and Innovation Global Practices (GPs) was an important factor behind the growth of agribusiness and value chain projects, interviews with country teams suggest obstacles to working across sectors. Collaboration was hindered, for example, by corporate incentives favoring certain GPs and task team leaders’ responsibilities, including through greater control of budgetary resources. There were also significant differences in perspectives between GPs on how best to address jobs objectives. Finally, the jobs agenda is not managed in a centralized manner within most client governments, which can contribute to a fragmented policy dialogue, which is not conducive to a more integrated approach within the World Bank, where different GPs may have different government interlocutors.

Based on the limited available data, the performance ratings of the closed projects in the evaluation portfolio were slightly better than those of the rest of the IDA portfolio. Given that the evaluation period starts in fiscal year 2015, there are relatively few closed projects with validated outcome ratings available. A comparison between jobs- and nonjobs-related closed investment projects with IDA financing indicates relatively better performance of the jobs-related projects. Two-thirds of jobs-related indicators in projects that have passed their Mid-Term Reviews were on track to meeting their targets, but, as noted, there were shortcomings in the many of the underlying indicators.

However, we can say little about the jobs outcomes of IDA-financed interventions. This is because the IDA strategy has not been successful at promoting better measurements of jobs outcomes, with more attention needed to enhance the systems for capturing labor market impact and outcomes. In most cases, the impact of interventions on jobs was only weakly captured through project development objective indicators, with project teams incentivized to focus more narrowly on what is directly attributable and tending therefore to measure the volume of outputs or number of beneficiaries instead. Examples include number of persons completing a skills training program or number of public employment centers upgraded.

Recommendations

On the basis of the evidence and findings presented in this evaluation, we make the following two recommendations:

Recommendation 1. IDA could strengthen the measurement of its contribution to the achievement of jobs objectives. This can be achieved through better and more relevant corporate-level indicators, better project monitoring and evaluation, and enhanced support for country statistical systems for measuring labor market outcomes. Implementation of this recommendation has the potential to significantly improve learning, adaptation, and accountability.

Recommendation 2. IDA could draw more systematically on jobs diagnostics to inform country-level operational engagement. This can be achieved by strengthening the ownership and use of jobs diagnostics and the integrated approach contained therein to inform policy dialogue, Country Partnership Framework priorities, and operational design. For example, Country Management Units could better incentivize cross-GP collaboration by drawing on diagnostics to establish an integrated and contextualized vision of policy priorities to be the basis of policy dialogue and project design.