Highlights

The World Bank Group adopted a new approach to disruptive and transformative technologies (DTT) in 2018, later merging it with the 2019 DTT Mainstreaming approach, and further developing it through the 2020 Mainstreaming Digital and Disruptive Technologies Initiative. Given that the Mainstreaming approach is relatively new, this evaluation focuses on the Bank Group’s preparedness for providing DTT support rather than on the outcomes of that support. It establishes a baseline and highlights specific areas for the Bank Group’s attention, action, and monitoring that can help it enhance the effectiveness of its future DTT support.

The main findings of this evaluation are twofold: (i) the Bank Group’s traditional areas of strength, such as its support for global public goods, honest broker role, production of quality advisory services and analytics, and ability to mobilize International Development Association resources and trust funds, have enabled its support for DTT for development; (ii) given the accelerating pace and complexity of technological change, the Bank Group is not yet sufficiently well prepared to help clients harness the opportunities and mitigate the risks posed by DTT, despite some areas of strength.

The Bank Group will need to seize opportunities where DTT offer the potential to achieve the twin goals more effectively or efficiently and play a particular role in addressing DTT risks. Support for DTT by the Bank Group may not be relevant in all contexts and at all times.

Context for World Bank Group Engagement

Disruptive and transformative technologies (DTT) have far-reaching implications for development. Traditional development models are being disrupted by the accelerating pace of technological change and the convergence of multiple technologies, among other things. What distinguishes the current DTT revolution from past technological revolutions (such as the industrial revolution) is the explosion of data. In 2015 and 2016 alone, more data were created than in all previous years combined.

Recognizing the implications of DTT for development, the World Bank Group adopted a new approach to DTT in 2018, later merging it with its 2019 DTT Mainstreaming approach and further developing it through the 2020 Mainstreaming Digital and Disruptive Technologies Initiative. The Bank Group’s approach encompasses five DTT corporate priorities (country diagnostics, agile regulations, digital connectivity, digital government, and skills and capabilities for the new economy and the role of education), the Bali Fintech Agenda (financial technology and digital entrepreneurship), and sectoral and regional programs (for example, Digital Economy for Africa Moonshot/Accelerate and MNA [Middle East and North Africa] Tech). The Bank Group aims to help clients harness the opportunities and mitigate the risks of DTT to accelerate progress toward achieving the twin goals of ending extreme poverty and boosting shared prosperity. The transition to a mainstreaming approach from earlier discrete areas of support marked a significant increase in the Bank Group’s ambition regarding DTT. Accordingly, the Bank Group’s definition of DTT broadened beyond digital technologies to include other technologies (such as robotics, artificial intelligence, the internet of things, biotechnology, 3D printing, solar power, and batteries) and the analog complements (such as policies, institutions, and skills). It also extended beyond the “application” of technology to development challenges (for example, using computers in the classroom) to the “response” to technology (for example, imparting socioemotional skills to equip people for future jobs that machines will not be able to perform). The Bank Group’s definition of DTT, however, lacks operational clarity and does not sufficiently distinguish among digital, disruptive, and transformative. To date, much of the Bank Group’s focus has been on digital technologies.

The Bank Group is implementing an array of DTT initiatives across all Regions and is engaged in innovative DTT initiatives, such as Development Data Partnership (formerly Data Collaboratives), Development Marketplace to Address Gender-Based Violence (in which the International Finance Corporation [IFC] also participated), Disruptive Technologies for Development Trust Fund, Geo-Enabling for Monitoring and Supervision, Geospatial Operations Support Team, the Information and Technology Solutions (ITS) Technology and Innovation Lab, Lighting Global, Social Development Data Lab, and IFC’s TechEmerge and ScaleX. Adding significance and urgency to the Bank Group’s DTT support are the DTT-related commitments for the 19th Replenishment of the International Development Association (IDA), the capital increase package, and opportunities offered by DTT to respond to the disruptions caused by the coronavirus (COVID-19).

Framework for Analysis

This evaluation sought to answer the following question: How well prepared is the Bank Group to help clients harness the opportunities and mitigate the risks posed by DTT?

The evaluation covered the World Bank and IFC (henceforth collectively referred to as the Bank Group) and drew on multiple sources. A literature review encompassing the organizational effectiveness literature and relevant Independent Evaluation Group evaluations identified the main preparedness dimensions for DTT support, which included dimensions of organizational ability and organizational willingness. Organizational ability was examined in terms of staff skills and mindsets, as well as internal processes and procedures. Organizational willingness was examined in terms of institutional culture and incentives. The relevance, timeliness, and effectiveness of the Bank Group’s support for DTT will also be influenced by how well that support is tailored to and affected by country context factors.

Thematic reviews were conducted of the five DTT corporate priorities, the Bali Fintech Agenda, and selected sectoral and regional programs. Semistructured interviews of 105 Bank Group staff probed their experiences with DTT and were analyzed using NVivo, a qualitative data analysis software. Interviews of 57 external stakeholders were also conducted.

Bank Group Preparedness

Given the accelerating pace and complexity of technological change, the answer to the evaluation question is that the Bank Group is not yet sufficiently well prepared to help clients harness the opportunities and mitigate the risks posed by DTT, despite some areas of strength.

Where the Bank Group Is Best Prepared

The Bank Group’s traditional areas of strength are also where it is best prepared to support DTT.

Supporting Global Public Goods

The Bank Group has a solid reputation for supporting global public goods, which increasingly use DTT. For example, the Global Facility for Disaster Reduction and Recovery supports the use of DTT through open-source technology and geospatial data-sharing platforms to help people worldwide access risk information. Furthermore, the Bank Group is recognized for its expertise in development data and is helping Bank Group operations take advantage of the DTT-enabled opportunities for data production, analysis, and use through initiatives such as the Development Data Partnership (formerly Data Collaboratives), Geo-Enabling for Monitoring and Supervision, and the Geospatial Operations Support Team.

Serving As an Honest Broker

The Bank Group’s reputation as an honest broker stems from its record of neutrality in dealing with public and private sector technology initiatives, its access to worldwide academic and commercial technological knowledge, and its prominent role in advising and negotiating with governments. By leveraging its honest broker role and adopting a problem-driven approach, the Bank Group—in its support for the Identification for Development program—has aimed to address the long-standing challenges of targeting the right beneficiaries and delivering multiple services through interoperable systems.

Undertaking Quality Analytical Work

The Bank Group’s ability to offer quality advisory services and analytics (ASA) has also enabled it to support DTT. ASA has equipped the Bank Group with actionable recommendations to support clients in putting their digital economies on a robust footing.

Providing Trust Funds and IDA Resources

The Bank Group’s ability to mobilize trust funds and IDA grants and credits has bolstered its DTT support. For example, in Benin, IDA financing (for the West Africa Regional Communications Infrastructure Program—Benin project) enabled the development of a competitive telecoms sector with open access to all providers of communications infrastructure. Furthermore, trust funds have supported World Bank task teams in piloting projects and gaining a better understanding of constraints. For example, in the Kenya Industry and Entrepreneurship Project, the use of trust funds to launch a competition among local start-up companies enabled the World Bank team to better understand issues such as local intellectual property rights.

Where the Bank Group Is Less Prepared

Links between the Bank Group’s DTT Support and the Twin Goals

Links between the Bank Group’s DTT support and the twin goals were assessed for two of the five Bank Group DTT corporate priorities, noting that all DTT corporate priorities are relevant for achieving the twin goals. The two selected DTT corporate priorities were (i) country diagnostics that help chart the new drivers of growth and (ii) skills and capabilities for the new economy and the role of education.

Country diagnostics. The Bank Group has given increased attention to the digital economy in a few recent country diagnostics and strategies. Of those referred to in the 2019 World Bank Mainstreaming Paper (World Bank 2019h), this evaluation found that discussion of the digital economy in the Morocco and Senegal Country Partnership Frameworks was informed by findings in their respective Systematic Country Diagnostics. However, this was not always the case. There has also been insufficient effort in the Systematic Country Diagnostics in linking the support for digital technologies to the twin goals (for example, when identifying key constraints). Furthermore, the newly introduced digital economy diagnostics have not adequately considered the broader development agenda and the twin goals. Contributing reasons include (i) limited participation by relevant Global Practices in the preparation of digital economy diagnostics and (ii) insufficient disaggregated data, such as internet usage by income group.

Skills and capabilities for the new economy and the role of education. In light of the changing nature of work, the World Development Report 2019 notes that lack of education is likely to be one of the strongest mechanisms for transmitting inequalities from one generation to the next (World Bank 2019m). It emphasized the need to invest early in twenty-first century skills to be prepared for the changing nature of work. DTT have a critical role to play in education, to equip workers and children for the job market of the future. For instance, the application of DTT can improve the delivery of education. In addition, the education system can facilitate a response to DTT by fostering the twenty-first century skills necessary for the job market of the future. The Bank Group’s top corporate initiatives in education—the Human Capital Project and the Learning Poverty initiative—spotlight “necessary” skills such as literacy and numeracy. The Bank Group has an opportunity to move the narrative forward from necessary skills to include twenty-first century “sufficient” skills, such as advanced cognitive skills, digital literacy, and socioemotional skills, in particular a growth mindset or the ability to “learn to learn” and adapt. Necessary and sufficient skills are critical for ensuring that a workforce is well prepared for the future labor market.

Staff Skills and Mindsets for DTT Support

The Bank Group’s fiscal year (FY)20–22 Human Resources Strategy presents key insights about staffing issues that confront the Bank Group, acknowledging the need to build the expertise required to cope with rapid technological change. This evaluation found that the Bank Group has yet to (i) identify the DTT-relevant skills that it needs; (ii) ensure that its information systems and databases contain the necessary information on its current DTT-relevant skills; and (iii) take action to fill any gaps. In interviews conducted for this evaluation, Bank Group staff noted that the number of existing staff with DTT-relevant skills was insufficient to meet client demand. In the education sector, staff referred to the EdTech Fellows program as an initiative that could provide lessons, positive and negative, on how to address skills shortages in DTT for development. Interviewees also noted the need to raise awareness and better leverage existing staff expertise and resources across different sectors. Furthermore, they reported that a mindset for continuous learning and adaptation would help enhance the effectiveness of the Bank Group’s DTT support. IFC interviewees perceived that DTT expertise was fragmented after the realignment of Telecoms, Media, and Technology teams, which initially diluted IFC’s ability to maintain thought leadership in DTT. The more recent absorption of the Telecoms, Media, and Technology group into the Infrastructure group and the Fintech team into the Financial Institutions Group, and the realignment and hiring of staff in the Disruptive Technologies and Funds group, in addition to the respective industries’ Upstream units (including specialists in artificial intelligence and machine learning), have sought to address gaps in DTT-relevant thought leadership and expertise. However, it is too early to assess the effectiveness of these measures.

Internal Processes and Procedures for DTT Support

Collaboration. The need for collaboration is underlined by the wide-ranging nature of DTT projects that cut across sectors and often involve both public and private institutions. The Independent Evaluation Group found examples of effective collaboration in providing DTT support in the Bank Group, for example the Development Data Partnership (formerly Data Collaboratives), the Digital Central Asia–South Asia program, Geo-Enabling for Monitoring and Supervision, and Identification for Development. However, interviewees from across Global Practices perceived that collaboration is insufficient among the different Global Practices, with instances of competition for task team leadership. The lack of collaboration between the World Bank and IFC is also an issue, because successful DTT-related outcomes require knowledge and inputs from both the public and private sectors.

Procurement (World Bank only). Procurement was a major constraint to the smooth implementation of DTT-related projects in the eyes of most World Bank interviewees. There is currently only fragmented guidance for complex technology projects, specifically on their associated risks (such as vendor lock-in, lack of interoperability of systems, potential loss of data ownership and lack of data privacy, limitations in technology-specific legislation and regulation, and mismatch between business processes and the requirements of the particular technology). The World Bank’s procurement systems have provided some flexibility for complex technology projects, for example the use of framework agreements for software maintenance or servicing. However, interviewees noted that there was insufficient guidance on how to apply the available flexibility in different situations, suggesting the need for such guidance. Reluctance to use the available flexibility may also result from misaligned incentives that encourage risk aversion.

Institutional Culture and Incentives for Risk Taking and Innovation

Harnessing DTT for development often demands innovation, which by definition is without precedent and inevitably risky. However, interviewees reported that DTT were treated in the same way as other sectors or themes, despite their fluid and fast-moving nature, and that relevant staff were not encouraged to keep at the cutting edge of technological advances and DTT trends. Furthermore, the levers at the disposal of the Bank Group (such as leadership signaling, questions asked in operational review meetings, and the performance management system) were not yet being effectively used to facilitate informed risk taking, learning from successes and failures, or innovation for DTT for development. Interviewees saw the operational review process as constraining innovation and creative solutions, given the general support for continuing on a familiar path rather than for breaking new ground. Interviewees reported that legal, procurement, and external relations departments often asked for precedents. Finally, risk taking—however informed and calculated—means failing at least some of the time, and interviewees said they would benefit from knowing how failure would be treated by Bank Group management.

Moving Forward: Directions of Travel

The Independent Evaluation Group presents—for the Bank Group’s consideration—possible directions of travel, including examples for each, that can help the Bank Group move forward in strengthening its preparedness for DTT for development.

1. Building on the Bank Group’s Existing Strengths

Mining DTT-generated development data to create even greater social value. DTT enable development data to be collected and analyzed cost-effectively, with high frequency, and at fine levels of granularity. This granularity can allow for personalizing services, for example tailoring learning to individual students in poor areas whose performance may be lagging and thus potentially helping reduce high dropout rates. Both the Bank Group’s World Development Report 2021 on data (World Bank 2021), as well as the recently established Data Governance body and its supporting Data Governance Steering Committee, could boost the World Bank’s ability to support development data. Optimizing the use of public and private data—both those made available by DTT and those that can be analyzed more effectively by using DTT—can help the Bank Group become a data-driven organization.

Leveraging the honest broker role by advising clients, especially on DTT risks. The Bank Group’s perceived neutrality positions it well to advise on DTT risks such as income inequality, data privacy, data security, and cybersurveillance. Attention to DTT risk is particularly important in Identification for Development systems, which are especially vulnerable to inadvertent data spills or willful misuse. Addressing DTT risks and adhering to the do no harm principle may require the Bank Group to navigate sensitive ethical and political issues and advise clients on them. Where it lacks the necessary technical expertise to help clients mitigate DTT risks, the Bank Group could use its convening power to bring together relevant experts.

Using explicit corporate metrics to track the implementation of the Bank Group’s DTT Mainstreaming approach. Explicit metrics—defined at the corporate level—can help track progress and enable timely course corrections, thereby facilitating learning and the successful implementation of the Mainstreaming approach.

2. Enhancing Bank Group Capabilities That Are Not Yet Its Forte

Addressing gender-differential impacts of DTT, beginning with ASA work. Although the Bank Group’s strategic documents recognize the gender-differential impacts of DTT, a review of gender and DTT for this evaluation found that just 7 percent of World Bank ASA in the information and communication technology (ICT) sector (ICT sector percentage weight of 50 percent or more) approved during FY15–18 were gender-relevant ASAs, a particular concern since it is in ASA that new opportunities to address the gender-differential impacts of DTT can be explored. (Gender-relevant ASAs were identified using a keyword search and desk review. Appendix C provides methodological details.) Furthermore, although 76 percent of the 51 World Bank ICT projects (ICT sector percentage weight of 50 percent or more) approved during FY15–19 had one or more gender-relevant flags, 37 percent had all three gender-relevant flags.

3. Developing New Strengths

Fostering a growth mindset in the Bank Group. Given the uncertainties created by technological change, the Bank Group could enhance its preparedness for DTT by ensuring a growth mindset that promotes continuous learning and adaptation among its staff. This would entail encouraging staff to recognize—and learn from—mistakes without fear of repercussion.

Systematically applying foresight and anticipation to inform and guide the Bank Group’s support of DTT for development. Given the accelerating pace of technological change and the associated uncertainty, foresight and anticipation have a particular role in DTT for development. Foresight and anticipation can enable the Bank Group to regroup and take on challenges in real time and thrive even in markedly changed circumstances. The Bank Group’s Country Partnership Framework is already applying foresight and anticipation to outline the Bank Group’s country strategy four to six years out. More specifically for DTT, the recent exploration of foresight and anticipation by the Bank Group’s Technology and Innovation Lab is a promising step.

Recommendations

This evaluation makes three recommendations:

Recommendation 1: Where DTT offer opportunities to make progress on the twin goals more effectively or efficiently, ensure that the Bank Group avails itself of those opportunities and addresses, in particular, the risks posed by DTT. This will entail, for example:

• Systematically identifying in both country diagnostics and digital economy diagnostics the opportunities and risks posed by DTT for achieving the twin goals;
• Consistently asking in operational review meetings (from Concept Note to approval and implementation stages) whether and how the use of DTT in operations can bring (or is bringing) effectiveness and efficiency gains in addressing the twin goals; and
• Strengthening cross-sectoral linkages and synergies between DTT and sectoral issues, including through enhanced collaboration across Global Practices and between the World Bank and IFC, particularly where different units support technological solutions with the same client.

Where appropriate, a complement to country and digital economy diagnostics could be a review of ongoing and pipeline projects for DTT opportunities, such as the one undertaken recently in Vietnam.

Recommendation 2: Build a Bank Group workforce with the skills required to harness DTT opportunities and mitigate DTT risks by identifying DTT-relevant skills, determining gaps in these skills, and filling these gaps. This will entail:

• New recruitment of staff or consultants to fill specific skills gaps, retraining of existing staff, outsourcing, secondments, external partnerships, or a combination of these;
• Ensuring a more efficient use of the skills of available specialized staff and a better bridging of technology expertise with that of sector specialists and task team leaders; and
• Supporting a growth mindset for continuous learning as noted in the Bank Group’s FY20–22 Human Resources Strategy.

Recommendation 3 (World Bank only): Improve the effectiveness and efficiency of World Bank procurement for complex technology projects. This will entail, for example:

• Strengthening procurement guidance for staff and borrowers, for early-stage project scoping as well as subsequent capacity development, on the identification, prioritization, and mitigation of the risks associated with complex technology projects. The guidance could cover: availing of existing flexibilities for complex technology projects, such as two-stage bidding; monitoring the extent to which World Bank staff teams adopt these flexibilities; and institutionalizing arrangements for technology project management.
• Ensuring that teams benefit from effective and efficient innovation while protecting the World Bank and the borrower from procurement-related reputational risks.
• Preparing a roster of the world’s leading experts on the procurement of complex technology projects and encouraging market consultations on technical requirements in the preparation of bidding documents.