Highlights

Jobs diagnostics addressed significant knowledge gaps.

Through a robust feedback loop with the International Development Association (IDA) cycle, jobs diagnostics informed the policy commitments of the 18th and 19th Replenishments of IDA.

The IDA jobs strategy and jobs diagnostics improved the articulation of jobs-related objectives, theories of change, and results frameworks in Country Partnership Frameworks.

The evolving IDA jobs strategy had a pronounced effect on engagements with countries affected by fragility, conflict, and violence, with an increased focus on youth employment.

The IDA jobs strategy did not successfully promote better measurements of jobs outcomes.

This chapter assesses the IDA jobs strategy’s analytical underpinnings and operational relevance. IEG examined whether policy commitments evolved over time to better reflect learning from country evidence and experience. IEG also evaluated the operational relevance of the policy commitments and the extent to which they were translated into country strategies. To assess the strength of the feedback loop between the IDA jobs strategy and country engagements, IEG (i) reviewed each of the three strategy components; (ii) carried out a structured review of 66 CPFs from a sample of 33 IDA countries to compare jobs-related objectives, results indicators, and theories of change before and after IDA17; (iii) performed content analysis of 22 jobs diagnostics; and (iv) interviewed country teams in 13 countries. Overall, we found that the IDA jobs strategy had a strong analytical underpinning and successfully stimulated country analytics and diagnostics to fill knowledge gaps. In turn, policy commitments adequately evolved across replenishments to reflect learning. The jobs strategy also had a discernable impact on country strategies, bringing jobs-related objectives into sharper focus and improving the coherence of the theories of change. However, the strategy failed to stimulate better measurement of jobs outcomes.

Analytical Underpinning

Jobs diagnostics helped address significant knowledge gaps. Jobs diagnostics were mandated by specific policy commitments in IDA17 and have provided an impetus for unpacking jobs constraints in IDA countries, filling an important knowledge gap. Conventional growth diagnostics and other country analytics did not focus on jobs, and much was unknown about the factors that influence jobs outcomes in IDA countries. The strategy thus called for a multisectoral approach and integrated diagnostics covering issues such as access to markets, inputs, capital, technology, skills, and matching supply and demand. A structured review of all 22 jobs diagnostics in IDA-eligible countries carried out for this evaluation found that the jobs diagnostics’ analysis of labor market trends was thorough, often broken down by sector, region, gender, and age. In all cases, there was a reference to gender gaps, and in 14 out of 22 cases, the jobs diagnostics also provided a thorough assessment of gender-differentiated outcomes and constraints to enhanced female labor force participation or formal employment. In most cases, youth employment challenges were well motivated and discussed, although in nine cases the analysis was rather shallow. The jobs diagnostics also offered a detailed analysis of the shifts in employment out of agriculture into services or manufacturing, as well as patterns of internal migration across regions and between rural and urban areas.

Most jobs diagnostics usefully incorporated recommendations for action. The structured review of 22 jobs diagnostics also analyzed the extent to which jobs diagnostics offered country-specific recommendations to realize jobs-related objectives. The review found that for active labor market policies, constraints to female employment, and barriers to youth employment, almost two-thirds of jobs diagnostics reviewed offered recommendations that were specifically tailored to country contexts and jobs challenges and to a specific strategy for prioritization. However, only 9 out of 22 jobs diagnostics included sector-specific recommendations to enhance labor demand, and only 2 jobs diagnostics divided recommendations by time horizon, giving a sense of how to combine short-run and long-run interventions to maximize outcomes. For example, under small and medium enterprise (SME) support, the Nepal jobs diagnostic suggests that in the short term, measures be taken to facilitate the growth of commercially sustainable firms by developing business support services and lower-hanging fruit to encourage financial access, whereas in the medium to longer term, deeper reforms could support foreign direct investment and its linkages with SMEs in addition to strengthening digital infrastructure and the enabling environment for digital solutions.

A robust feedback loop existed between evidence from jobs diagnostics and subsequent IDA cycles. The policy commitments adequately evolved over time to reflect learning from country evidence and experience. The jobs diagnostic was seen as necessary, given that traditional growth diagnostics and other country analytics did not focus enough on jobs. For example, a paper synthesizing early lessons from 15 pilot jobs diagnostics highlighted the fragmentation of the private sector in IDA countries, which are dominated by self-employment and household enterprises and heavily dependent on low-productivity agriculture (Merotto, Weber, and Aterido 2018). The findings informed the IDA18 jobs strategy. They clarified IDA’s jobs-related objectives, adding a specific objective on inclusive jobs, balancing the need to promote formal sector jobs and raising income in the informal sector, and focusing attention on the importance of raising productivity in agriculture and promoting agribusiness. Jobs diagnostics also revealed issues with labor force data and evidence, which was the subject of several IDA18 policy commitments to improve data availability and support evidence-based project design. In turn, IDA19 pivoted toward targeted policy commitments to address specific constraints that had been found to be particularly challenging in jobs diagnostics and other country analytics. For example, IDA19 committed to having at least 66 percent of agriculture, and agribusiness projects in IDA countries include support for participation in value chains that were seen to have high potential for growth and job creation.

Operational Relevance

The IDA jobs strategy and jobs diagnostics improved the articulation of jobs-related objectives, theories of change, and results frameworks in CPFs. IEG conducted a before-and-after analysis on five dimensions of the CPF for 33 IDA-eligible countries. For each country, we compared the articulation of jobs-related CPF objectives, theories of change, results framework indicators, youth, and gender prioritization in country strategies designed before and after IDA17. Moreover, the CPFs now have clearer and more explicit statements of jobs-related objectives (22 out of 33) than before IDA17 (15 out of 33). The IDA strategy also nudged a better articulation of both direct and indirect pathways to jobs outcomes. Before IDA17, CPFs that had a clear and coherent theory of how the World Bank’s interventions would contribute to better jobs were rare (2 out of 33 had a fully laid-out theory of change, and 10 had a partial one), whereas they are now much more common (15 have a fully laid-out theory of change, and 8 have a partial one). For example, in Sierra Leone, the FY21 CPF placed jobs at its center, through a theory of change linking the entire portfolio to employment generation.

The specification of how the portfolio of interventions can address barriers to more, better, and more inclusive jobs is particularly well done in countries with a jobs diagnostic. Previously, support for jobs had been implicit—usually taking place through business environments and education reforms, as in Bangladesh, or as support for a more transparent, rules-based relationship between the state and the private sector, as in the Kyrgyz Republic, or improvements in agricultural productivity, as in most other countries. The improved theories of change are also more routinely reflected in results frameworks, which now more often include indicators to capture progress toward more or better jobs (in 21 cases in the post-IDA17 CPF versus 15 prior). However, most of these indicators continue to be output, rather than outcome, oriented as discussed in this chapter. There was no significant change in the prioritization of gender and youth as cross-cutting themes because this was already common before IDA17.

The IDA jobs strategy had a pronounced effect on engagements with countries affected by fragility, conflict, and violence (FCV), especially with an increased focus on youth employment. Before 2014, the World Bank lacked a concrete framework to address inclusive jobs constraints in fragile and conflict-affected states (World Bank 2014b). To address this issue, IDA17 policy commitments under the FCV pillar led in 2015 to the development of a specific integrated jobs strategy for FCV countries (box 2.1). The development of a specific jobs framework for FCV countries came out of the realization that carrying out full-fledged jobs diagnostics in these countries was impractical because of the data requirements. The framework recognized the importance of jobs for social cohesion and strengthened the focus on jobs-related objectives in these countries. Out of $21.4 billion in IDA financing via (nonadditional financing) IPF, $5.2 billion (32 percent) went to countries that were on the World Bank’s FCV lists at the time of project approval. This proportion was slightly lower, at 27 percent of total financing, in the nonjobs-related IDA-financed portfolio of nonadditional financing IPF during the same period. The jobs-related projects in countries classified as fragile and conflict-affected situations tended to have a higher share of total commitments allocated to jobs components as compared with non-FCV countries. This could explain why, out of 10 IDA-eligible countries with no jobs interventions during the evaluation period, 6 were on the FCV list.

The jobs portfolio in FCV countries placed greater emphasis on operations designed to improve youth employment and the economic inclusion of displaced people. The strategy highlighted the importance of youth economic activity for stability, and this imperative was translated into the design of operations that targeted youth. The share of youth employment intervention in FCV countries is also slightly higher than in the rest of the IDA portfolio. Of jobs-related projects in FCV countries, 68 percent had at least one youth-focused intervention (versus 46 percent of projects in non-FCV countries). The portfolio also contained 14 projects that specifically target refugees and displaced persons, with a focus on economic integration into host communities. For example, the Kenya Development Response to Displacement Impacts Project (P166266, ongoing) and the Socio-economic Inclusion of Refugees and Host Communities in Rwanda Project (P164130) aim to improve access to basic social services, expand economic opportunities, and enhance environmental management for communities hosting refugees. In Kenya, this includes activities supporting capacity building, technology access in farm and nonfarm households among host communities, and grants that are expected to support higher incomes among beneficiaries. In Rwanda, this includes support to market and educational infrastructure accessible to displaced persons, as well as jobs skills training for both refugees and host communities to improve employability. However, as explained in the Results Measurement section in chapter 2, because of measurement issues, it was sometimes unclear whether these operations led to improved outcomes for these groups.

Results Measurement

The IDA jobs strategy did not successfully promote better measurements of jobs outcomes. There is an inherent tension between the IDA jobs strategy’s ambition of better capturing jobs outcomes and the systems and incentives underlying results frameworks. IDA Results Measurement System Tier 2 indicators are meant to capture intermediate outcomes from IDA financing. The main Tier 2 indicator related to jobs is an output indicator (number of beneficiaries in IDA countries of jobs-focused interventions). It does not adequately capture progress on addressing constraints to jobs creation or measuring jobs quality and sustainability. Formal Operations Policy and Country Services guidance is that “an operation’s objective focuses on outcomes for which the project can reasonably be held accountable. It neither encompasses higher-level objectives beyond the purview of the project, nor is it a restatement of the project’s activities.” Interviews with task teams suggest that this inconsistency in internal expectations disincentivizes teams to capture jobs outcomes or invest in systems that can measure them. Moreover, unlike gender-disaggregated measurement, IDA Results Measurement System Tier 3 indicators do not capture how well IDA operations articulate and track contributions to jobs outcome. There are also technical difficulties in the complex and data-intensive task of estimating the direct and indirect impact of IDA interventions on jobs created. IDA19 committed to experimenting with model-based estimation of jobs impact on 20 projects. Twenty-three pilots were completed in FY22, but the methodology has not been used for project selection or as a basis for project design and monitoring.

Overall, we know little about the jobs outcomes of IDA-funded interventions or their effect on welfare and inclusion. In most cases, the impact of interventions on jobs was only weakly captured through project development objective indicators, with projects tending instead to measure the volume of outputs or number of beneficiaries. Table 2.1 shows the shares of a sample of results framework indicators relevant to jobs and whether they captured outputs or outcome in a stratified random sample of indicators based on text searches. The share of outcome-level indicators is overall quite low (only one-third), except for indicators that seek to enhance youth employment, which tend to capture outcomes (48 percent) more systematically.

The outcome-orientation of jobs objectives in results frameworks was particularly weak with respect to access to finance, value chain, and agribusiness projects. Only a few, such as the Integrating Innovation for Rural Competitiveness in Honduras—COMRURAL II project, measured jobs created or sustained and instead focused on either increases in yields or technology adoption or the value of sales, exports, and financing mobilized. Value chain projects similarly tended to focus on outcomes for firms rather than workers, although the Bangladesh Export Competitiveness for Jobs project, the Democratic Republic of Congo SME Development and Growth Project, and the Empowering Women Entrepreneurs project captured wage growth and jobs created. The Emergency Support for Micro, Small, and Medium Enterprises Project for the Kyrgyz Republic and the Access to Finance for Recovery and Resilience Project for Rwanda also measured the survival and employment rates of supported relative to unsupported firms. Conversely, impacts on individuals, whether employment or income, were better tracked through skills development and livelihoods programs (although not always and, in the former case, generally only up to six months after the provision of assistance).

Source: Independent Evaluation Group.

Note: Classification of nine indicators as measuring outcomes or outputs was omitted for intervention types where the sample size was less than 10. LIPW = labor-intensive public works; MSME = micro, small, and medium enterprise; n.a. = not applicable; TVET = technical and vocational education and training.

Conversely, the dedicated efforts to incentivize youth-focused and gender-disaggregated measurements of IDA jobs interventions paid off. Reinforcing commitments made under the gender strategy, IDA policy commitments set out to increase gender-disaggregated data in projects, including jobs-related operations. For example, in IDA17, at least seven Tier 3 indicators tracked how well projects articulated or tracked gender results. As shown in table 2.1, the majority of sampled indicators in projects with female employment interventions were relevant to jobs, and almost half were outcome oriented. These monitoring and measurement efforts were complemented by significant investments in generating impact evaluation results, particularly through Regional Gender Innovation Labs mobilized to carry out more than 70 impact evaluations—most of them in IDA-eligible countries in Africa—to test various interventions seeking to close the gender gap in earnings, productivity, assets, and agency. Learning from impact evaluations was evident in the portfolio, with several examples of successful interventions replicated in other IDA countries. For example, a 2016 impact evaluation of noncognitive skills development in Togo demonstrated that women who received innovative entrepreneurial training—focused on personal initiative—saw profits increase by 40 percent, compared with 5 percent for those who had received traditional business training. This intervention has since been replicated in projects in Ethiopia, Mauritania, and Mozambique.