The World Bank Group adopted a new approach to disruptive and transformative technologies (DTT) in 2018, later merging it with the 2019 DTT Mainstreaming approach, and further developing it through the 2020 Mainstreaming Digital and Disruptive Technologies Initiative. The transition to a mainstreaming approach from earlier discrete areas of support marked a significant increase in the Bank Group’s ambition regarding DTT.
DTT-related commitments pertaining to the 19th Replenishment of the International Development Association and the capital increase package, as well as the opportunities DTT offer to respond to the disruptions caused by the coronavirus (COVID-19), add significance and urgency to the Bank Group’s DTT work.
Consistent with the Bank Group’s use of the term, this evaluation adopts a broad interpretation of DTT and covers (i) support to digital technologies (such as the internet, mobile phones, smartphones, and Wi-Fi), and to other technologies (such as robotics, artificial intelligence, the internet of things, nanotechnology and biotechnology, 3D printing, batteries, drones, solar panels, and self-driving vehicles); (ii) enabling policies, institutions, and skills; (iii) relevant lending and nonlending operations in all sectors (not just in the digital development and information and communication technology sectors); and (iv) both the “application” of technology (for example, using computers in the classroom) and the “response” to technology (for example, imparting socioemotional skills to equip people for future jobs that machines will not be able to perform).
The world is experiencing rapid technological change with far-reaching implications for development. The accelerating pace of technology diffusion, the convergence of multiple technologies, and the emergence of global platforms are disrupting traditional development models (Pauwels 2019; World Bank 2019h). Disruptive and transformative technologies (DTT) are already reshaping the way goods and services are produced and consumed and have profound implications for the functioning and dynamics of the global economy (Dobbs, Manyika, and Woetzel 2015; Manyika and others 2017). At the national level, traditional development paths may no longer be viable, at least in some developing countries, as they run out of low-skilled manufacturing opportunities sooner and at much lower levels of income compared with the experience of early industrializers (Hallward-Driemeier and Nayyar 2018).
DTT offer new opportunities for reducing poverty and promoting shared prosperity. Business as usual is unlikely to get countries to the Sustainable Development Goals by 2030 (Kramer, Agarwal, and Srinivas 2019). DTT can improve people’s lives and transform economies, governments, and societies, for example by enhancing connectivity among markets around the globe, opening up new jobs and sources of livelihood, and improving the delivery of social and financial services (Brynjolfsson and McAfee 2014; Dobbs, Manyika, and Woetzel 2015; Manyika and others 2017). The United Nations Secretary General’s High-Level Panel on Digital Cooperation (2018) advanced proposals to strengthen cooperation on digital issues among governments, the private sector, civil society, international organizations, academia, the technical community, and other relevant stakeholders. Box 1.1 presents examples of development opportunities created by DTT.
Box 1.1. Disruptive and Transformative Technologies Have Created New Development Opportunities
Low-Tech Development Solutions: Smartphones and Audio
- The World Bank’s Geo-Enabling for Monitoring and Supervision is a client-focused capacity-building program that enables local stakeholders to leverage simple and low-cost digital tools (“pocket science”) for customized monitoring and evaluation, remote supervision, real-time risk monitoring, and portfolio mapping for coordination across projects and partners (World Bank 2019k).
- Amplio Network, a nonprofit organization, is sharing knowledge with the world’s hardest-to-reach communities through an easy-to-use “talking book” audio device designed to provide information to people who cannot read and who live in places where there is no electricity or network and collect their feedback (Brodnig and others 2020).
High-Tech Development Solutions: Artificial Intelligence
- Deep-learning algorithms are diagnosing retinopathy in patients living in rural India, where there is a shortage of ophthalmologists.
- Portable genomic sequencers are bringing the lab to the jungle, allowing for the diagnosis of the Ebola virus in “hot zones.”
- Open Innovation Lab in Shenzhen, China, young inventors have designed wearable devices that rely on image recognition to help farmers detect diseases on crops.
- Companies like Zipline are using artificial intelligence in autonomous drones to deliver critical medical supplies, such as vaccines, to rural hospitals in Africa.
- The United Nations Children’s Fund is collaborating with the Massachusetts Institute of Technology on deep-learning expertise to simulate images of major global cities in ruin to promote empathy and connection with the suffering of those who have experienced bombing, loss, and war.
- The combined optimization of biometric, genomic, behavioral, and physical systems data is giving rise to “affective computing”—algorithms that can successfully analyze, nudge, and communicate with people. Affectiva, an affective computing company, has developed Peppy Pals, a series of educational apps that teach children about social and emotional intelligence by learning from situations in an online world.
- The World Development Report 2019 notes that Ant Financial (of the Alibaba Group) makes loans using a “3-1-0” online lending model based on big data and artificial intelligence, which involves a three-minute application process and a one-second processing time, with zero manual interventions (Pauwels 2019; World Bank 2019m).
- India’s Mobile-Based Surveillance Quest Using Information Technology is a smartphone-based platform that enables better outbreak monitoring and real-time responses for malaria (Pathways for Prosperity Commission 2019a, 2019b).
Source: Independent Evaluation Group.
DTT also pose risks for reducing poverty and promoting shared prosperity. At the macro level, new technologies may reduce job opportunities for the less skilled. According to recent studies by the International Monetary Fund (2018) and the Organisation for Economic Co-operation and Development (2018), technological change likely has contributed to widening inequalities in earnings, shifting labor demand toward high-skilled occupations, and promoting automation over routine work. Concentrating market power in the hands of digital and data service providers, for example, may give rise to new policy and regulatory issues concerning monopolies, competition, accountability, data privacy, and taxation (Lee 2018). Furthermore, deep-learning systems (systems inspired by the structure and function of the brain or neural networks) can drastically intensify the nature and scope of cyberespionage and cyberattacks within smart cities (Pauwels 2019) and enable cybersurveillance by governments for political ends. New technologies also bring the risk that a few powerful states will shape the digital future for everyone else (Pathways for Prosperity Commission 2019a). Box 1.2 highlights the risks posed by DTT.
Box 1.2. Risks Posed by Disruptive and Transformative Technologies
- Skills and income inequality. The World Development Report 2016 pointed out that with the changing nature of jobs, those who can acquire additional skills will switch to better-paid nonroutine occupations; they will be the winners, and others will be the losers (World Bank 2016e). Automation poses a greater risk for women as they tend to be overrepresented in routine work, although this depends on their level of education (Brussevich and others 2018).
- Platform firms and income inequality. By operating in regulatory gray zones, platform firms can avoid taxes, creating huge wealth for a few with data collected from the many, thereby creating monopolistic tendencies and exacerbating income inequality.
- Algorithms as “weapons of math destruction.” Algorithms can be opaque, unregulated, and uncontestable and may reinforce bias: “If a poor student cannot get a loan because a lending model deems him too risky (by virtue of his zip code), he’s then cut off from the kind of education that could pull him out of poverty, and a vicious spiral ensues” (O’Neil 2016).
- “Internet of bodies,” data privacy, and cybercrime. Most corporate artificial intelligence platforms already have access to individuals’ online behaviors, relationships, health, and emotional states. But soon enough, these platforms will acquire baseline information about people’s vital signs, organs, and genomes. The “internet of bodies” and simpler digital technologies all have implications for data privacy and cybercrime (Pauwels 2019; USAID 2018b).
- Internet’s environmental footprint. Observers are increasingly concerned about the internet’s environmental footprint. They point out, for example, that mining of blockchain-based digital currencies is hugely energy intensive (see Randeep Sudan comments in Stanley 2017). Concerns about the effects of electromagnetic radiation on people, plants, bee colonies, and frogs around the globe were presented at the United Nations 2018 Forum on Science, Technology and Innovation (Singer 2014).
- Social media, aspirations, envy, and violence. The World Development Report 2019 notes that technology, in particular social media, affects the perception of rising inequality in many countries and heightens feelings of alienation and envy, which can in turn incite discontent, conflict, and violence (World Bank 2019m). Women and girls are more likely to experience threats of intimidation, harassment, defamation, violence, surveillance, or some combination of these than are men and boys (World Wide Web Foundation 2015).
The explosion of data distinguishes the current DTT revolution from past technological revolutions (such as the industrial revolution): In 2015 and 2016 alone, more data were created than in all previous years combined (Goldberg 2020). The recent explosion of data has largely come from private sources, such as mobile phones, electronic transactions, and satellites. Such data are collected cost-effectively, with higher frequency, and at fine levels of granularity, typically by private sector actors. Such data has fueled innovation in the private sector. According to the World Development Report 2021, development policy has traditionally relied on public data collected by governments, which offer better coverage of the population of interest but can also be costly, collected infrequently, and lack granularity (World Bank 2021).
The World Bank Group has adopted a new approach to DTT in recognition of the opportunities DTT offer and the risks they pose. The Bank Group outlined its new approach to DTT in a 2018 Development Committee paper, “Disruptive Technologies and the World Bank Group: Creating Opportunities—Mitigating Risks” (World Bank 2018b), which was subsequently merged with the 2019 Development Committee paper “Mainstreaming the Approach to Disruptive and Transformative Technologies at the World Bank Group” (henceforth called the “Mainstreaming paper”; World Bank 2019h). The new Mainstreaming approach aims to harness the opportunities and mitigate the risks posed by DTT to accelerate progress toward the Sustainable Development Goals and the Bank Group’s twin goals of ending extreme poverty and boosting shared prosperity. The Mainstreaming paper was accompanied by the Bali Fintech Agenda, jointly prepared by the World Bank and the International Monetary Fund (World Bank 2018g). More recently, the Bank Group’s 2020 Program Document for the Mainstreaming Digital and Disruptive Technologies (MDDT) Initiative further develops the Mainstreaming approach, mainly outlining the institutional arrangements for DTT (a Network, a Technical Working Group, and a Secretariat) within the Bank Group (World Bank 2020d).
The Mainstreaming approach to DTT builds on previous technology strategies, while implying a significant increase in ambition. The Bank Group’s past technology engagements encompassed support to science and technology, research and development, and innovation. Specific technology strategies have existed since 2002, mainly for the information and communication technology (ICT) sector. In transitioning from discrete areas of support to mainstreaming, the Bank Group’s Mainstreaming approach extends beyond ICT to other sectors and covers the enabling policies, institutions, and skills. Appendix H provides details of the evolution of the Bank Group’s approach to DTT.
The Independent Evaluation Group (IEG) definition of DTT is consistent with that of the Bank Group. Box 1.3 provides further detail of the definition, and box 1.4 lists the various programs and units engaged in DTT work across the Bank Group.
Box 1.3. World Bank Group and Independent Evaluation Group Definition of Disruptive and Transformative Technologies
First, the World Bank Group’s Mainstreaming paper notes that disruptive and transformative technologies (DTT) are “often based on digital technologies and products,” but that these “go far beyond connectivity and the potential of the internet” (World Bank 2019h, 4). Second, beyond digital technologies, DTT include technologies such as robotics, artificial intelligence, the internet of things, nanotechnology and biotechnology, 3D printing, batteries, drones, solar panels, and self-driving vehicles. Third, DTT also include technology’s analog complements (such as enabling policies, institutions, and skills; World Bank 2016e). Fourth, in addition to the “application” of technology to development (for example, using computers in the classroom), DTT also cover the “response” to technology, as in the case of education (for example, imparting socioemotional skills to equip people for future jobs that machines will not be able to perform). Fifth, the Bank Group recognizes the close relationship between DTT and innovation. Innovation both impacts and influences DTT and results from them: innovation has created DTT solutions such as online education and telemedicine, and DTT have led to innovations such as artificial intelligence and the internet of things. Furthermore, the links between DTT and innovation are recognized in the East Asia and Pacific Region’s DTT framework called Connect, Harness, Innovate, Protect (CHIP). Sixth, DTT cover both new uses of old technologies (for example, mobile phones) and new technologies and business models. Seventh, the magnitude of the “step change” in disruptive technologies or the degree of transformation that defines transformative technologies were not specified in the Bank Group’s Mainstreaming paper, leaving the terms disruptive and transformative ambiguous. Finally, the Mainstreaming paper presented digital connectivity and agile regulation as two of the Bank Group’s five DTT corporate priorities, implying that the Bank Group conceives of DTT as an umbrella term that subsumes digital technologies.
Source: Document reviews by the Independent Evaluation Group.
Box 1.4. Disruptive and Transformative Technologies Programs and Units in the World Bank Group
Key World Bank programs and units focused on disruptive and transformative technologies (DTT) include the following: • AgObservatory • Artificial Intelligence Lab • Blockchain Lab • Cybersecurity workstream • Data Collaboratives • Data Council • Data Development Group • Data Lab • DECAT • DEC Data • Digital Technology for Development • Disruptive KIDS (Knowledge, Information and Data Services Helpdesk) • EdTech • Fintech Coordination Group • Gender Innovation Lab • Geo-Enabling Initiative for Monitoring and Supervision • Geospatial Operations Support Team • GovTech • Information and Technology Solutions (ITS) Geospatial • ITS Innovation Lab • Open Learning Campus • Social Development Data Lab • Sustainable Development Remote Preparation and Supervision Group • Technology and Innovation Lab• World Bank Data Catalog • workstream on data and privacy (piloted in Middle East and North Africa and in Europe and Central Asia) • regional DTT initiatives such as Digital Central Asia–South Asia (Europe and Central Asia), Connect, Harness, Innovate, Protect—CHIP (East Asia and Pacific), Digital Economy Country Diagnostics and advisory services and analytics (South Asia), Digital Economy for Africa Moonshot/Accelerate (Africa), Digital Transformation Program (Latin America and the Caribbean), and MNA Tech (Middle East and North Africa) • leveraging digital technologies to support the Jobs and Economic Transformation Agenda in International Development Association countries • several DTT activities linked to COVID-19 • data programs and units covering data collection; data storage, management, and sharing; data analysis; and work and industry.
Key International Finance Corporation units and programs focused on DTT include the following: • the Telecoms, Media, and Technology group within the Infrastructure and Natural Resources Department • the Fintech team in the Financial Institutions Group • the Disruptive Technology and Funds Group • TechEmerge • ScaleX.
In addition, the International Finance Corporation has created Upstream units for each of the industry departments, including those focusing on DTT, to support market creation initiatives and pipeline development for the International Finance Corporation.
Source: World Bank Group data (not evaluated by the Independent Evaluation Group).
The Bank Group has adopted the build, boost, and broker value proposition to mainstream its DTT approach and is pursuing five corporate priorities, the Bali Fintech Agenda priorities, and sectoral and regional programs. The five corporate priorities are (i) country diagnostics, (ii) agile regulations, (iii) digital connectivity, (iv) digital government, and (v) skills and capabilities for the new economy and the role of education. Following the Bali Fintech Agenda, the Bank Group is also supporting fintech and digital entrepreneurship. Sectoral and regional programs to support the Mainstreaming approach include the Digital Economy for Africa (DE4A) Moonshot/Accelerate and MNA (Middle East and North Africa Tech). Appendix B provides details.
The Bank Group is implementing an array of DTT initiatives across all Regions. These initiatives include lending and nonlending work: (i) in Europe and Central Asia, through the Digital Central Asia–South Asia (CASA) optical fiber cable program; (ii) in East Asia and Pacific, through the Connect, Harness, Innovate, Protect (CHIP) framework; (iii) in South Asia, through the Digital Economy Country Diagnostics; (iv) in Africa, through the DE4A Moonshot/Accelerate with the African Union; (v) in Latin America and the Caribbean, through the Digital Transformation Program with the Organization of Eastern Caribbean States countries; and (vi) in Middle East and North Africa, through MNA Tech, focused on the digital transformation of governments and economies as well as regional integration. Bank Group initiatives also include a significant pipeline of country and regional initiatives.
The Bank Group is also engaged in innovative DTT initiatives. Innovative DTT initiatives include the Development Data Partnership (formerly Data Collaboratives), Development Marketplace to Address Gender-Based Violence (in which the International Finance Corporation [IFC] also participated), Disruptive Technologies for Development Trust Fund, Geo-Enabling for Monitoring and Supervision (GEMS), Geospatial Operations Support Team, Information and Technology Solutions (ITS) Technology and Innovation Lab, Lighting Global (including Lighting Africa and Lighting Asia), and Social Development Data Lab. IFC’s innovative DTT initiatives include TechEmerge and ScaleX (which increases access to funding for women entrepreneurs).
DTT-related commitments in the 19th Replenishment of the International Development Association (IDA) and the capital increase commitments add significance to the Bank Group’s DTT support. These include several policy commitments relating to technology that the Bank Group will need to meet, for example boosting broadband penetration and supporting digital financial services and entrepreneurship; supporting women’s digital skills and resulting access to higher productivity jobs and increased access to ICT services; and supporting the use of field-appropriate digital tools and analysis in fragile and conflict-affected situations (IDA 2020). Regarding the capital increase commitments, Bank Group management has undertaken to further leverage smart technology solutions (World Bank 2018e).
More recently, by bringing to light the importance of social distancing and travel restrictions to contain pandemic diseases, the coronavirus (COVID-19) has elevated the urgency of location-neutral solutions, for which DTT are especially suited. The Organisation for Economic Co-operation and Development and the World Bank are collaborating to use DTT (for example, mobile apps, artificial intelligence [AI], and big data) to aid the COVID-19 response around the world. The four directly health-related uses of DTT are (i) outbreak spread monitoring; (ii) prevention and symptom tracking; (iii) contact tracing; and (iv) enforcement of containment measures (Amaral, Vranic, and Lal Das 2020). Beyond these uses, the Bank Group has experience in other areas—such as fintech; digital IDs; technology-enabled operations in fragility, conflict, and violence contexts; and distance education and online learning—that can also be deployed to respond to COVID-19. IFC has also sought to leverage its venture capital portfolio in HealthTech and EdTech. The Bank Group has provided $160 billion in financing through June 2021, including to support digital foundations and applications (appendix J). This significant commitment for the pandemic response demonstrates the Bank Group’s preparedness to quickly address exogenous shocks. An IEG evaluation is planned to assess the quality and relevance of the response.
About This Evaluation
This evaluation sought to answer the following question: How well prepared is the Bank Group to help clients harness the opportunities and mitigate the risks posed by DTT? Key aspects of the evaluation’s focus and scope are as follows. First, the focus of the evaluation is on the preparedness of the Bank Group to provide DTT support rather than on measuring the effectiveness or outcomes of that support—a deliberate choice given the newness of the Bank Group’s Mainstreaming approach. Second, the term DTT refers to DTT for development. Third, the evaluation does not include an assessment of the Bank Group’s own use of computer hardware and software or of its own information technology (IT) systems. Fourth, the evaluation’s analysis of education, procurement, and gender covers the World Bank only. Fifth, in this report the term Bank Group is used to cover the World Bank and IFC only, not the Multilateral Investment Guarantee Agency or the International Centre for Settlement of Investment Disputes, which are not part of this evaluation.
IEG identified the preparedness dimensions from the organizational effectiveness literature and relevant IEG evaluations and organized these into two groupings: (i) organizational ability (in terms of staff skills and mindsets, as well as internal processes and procedures such as collaboration and procurement) and (ii) organizational willingness (in terms of institutional culture and incentives for risk taking and innovation)., ,
These dimensions were examined to assess how well prepared the Bank Group was to help clients harness the opportunities and mitigate the risks posed by DTT. The overall framework for analysis informing this evaluation was that preparedness dimensions relating to organizational ability and organizational willingness influence the Bank Group’s support to DTT with regard to both its five DTT corporate priorities, fintech, and digital entrepreneurship, as well as its sectoral and regional programs. The relevance, timeliness, and effectiveness of the Bank Group’s support for DTT will also be influenced by how well the Bank Group support for DTT is tailored to and affected by country context factors. Figure A.1 in appendix A illustrates this.
Specifically for IFC, the assessment of DTT preparedness focused mainly on the corporate priorities of the Mainstreaming paper and interventions from relevant key departments. These key departments were as follows: for the digital connectivity corporate priority, the Telecoms, Media, and Technology (TMT) group in the Infrastructure and Natural Resources Department; for fintech and digital entrepreneurship, the Financial Institutions Group and the Disruptive Technology and Funds group. Activities of other units in IFC’s evolving organizational structure, notably the Upstream units created in fiscal year (FY)20, were too recent to be the subject of this evaluation.
The main evaluation instruments included the following:
- A review of the organizational effectiveness literature and existing IEG evaluations.
- Thematic reviews covering the Bank Group’s DTT corporate priorities, fintech, and digital entrepreneurship as well as selected sectoral and regional programs.
- A review of project documents and advisory services and analytics (ASA) relating to all DTT-related World Bank projects in education and gender over the past five years.
- A review of tech-informed Systematic Country Diagnostics (SCDs) and Country Partnership Frameworks (CPFs).
- Document reviews of lending and ASA for a purposive sample (selected based on interviews and database analysis) of digital connectivity in the World Bank and IFC (investment projects), electronic government services (especially those pertaining to social protection and digital ID), fintech (in both the World Bank and IFC), and digital entrepreneurship operations.
- A review of the Bank Group’s information systems and databases—in collaboration with the Development Economics, Development Data Group (DECDG)—to assess the staff skills for DTT work.
- Semistructured interviews covering 105 Bank Group staff, including World Bank and IFC team leaders and investment officers, as well as 57 external stakeholders, including World Bank and IFC clients. The 105 Bank Group staff interviews were analyzed using NVivo, qualitative data analysis computer software. Details of the NVivo analysis and other methodological aspects are presented in appendix A.
There is no official Bank Group operational list that identifies DTT-related projects and ASA (except for Digital Development or ICT projects and ASA). Consequently, IEG identified DTT-related projects and ASA in consultation with the relevant Global Practices. It did so for two Bank Group priorities that have strong significance for the twin goals: education (the Bank Group’s DTT corporate priority skills and capabilities for the new economy) and Gender (Bank Group Global Theme).
Given that the Bank Group’s Mainstreaming approach for DTT is relatively new, this evaluation focused on Bank Group preparedness for providing DTT support rather than on the outcomes of that support. Most of the Bank Group’s DTT-related operations identified in the Development Committee papers are incipient or incomplete, meaning that effectiveness or outcomes, which are among the key measures of Bank Group preparedness, do not yet exist (World Bank 2018b; World Bank 2019h). In this context, the evaluation establishes a baseline and highlights specific areas for the Bank Group’s attention, action, and monitoring that can help it enhance the effectiveness of its future DTT support to clients.
Bank Group data related to DTT preparedness (such as on staff skills and mindsets, collaboration, procurement, and institutional culture and incentives) are, at best, incomplete, requiring IEG to draw on staff interviews. Staff interviews had the advantage of helping assess not only how things looked on paper but also how staff perceived these things in practice. If staff perceived that the Bank Group had not equipped them (for example, with appropriate skills and knowledge), empowered them (for example, through adequate decision-making power to innovate and chart a new course), or supported them (for example, when they took risks and failed), it would not matter how good things looked on paper. NVivo was used to systematically analyze staff views.
Chapter 2 focuses on key Bank Group enablers of DTT support and the links between this support and the twin goals. The links with the twin goals were assessed for two select Bank Group DTT corporate priorities: (i) country diagnostics that help chart the new drivers of growth (corporate priority 1) and (ii) skills and capabilities for the new economy (corporate priority 5) and the role of education. Chapters 3 and 4 examine the Bank Group’s preparedness regarding organizational ability in terms of staff skills and mindsets (chapter 3), and internal processes and procedures, specifically collaboration and procurement (chapter 4). Chapter 5 assesses the Bank Group’s preparedness regarding organizational willingness in terms of the Bank Group’s institutional culture and incentives for risk taking and innovation. Chapter 6 presents—for the Bank Group’s consideration—directions of travel that can help the Bank Group move forward in strengthening its preparedness for DTT for development. Chapter 7 concludes and makes three recommendations.
- The World Development Report 2013 pointed out how India and other countries set up high-tech enclaves despite poor overall information communication technology infrastructure (World Bank 2012b, 55–57). See also Peralta-Alva and Roitman (2018) for a comparison.
- The United Nations High-Level Panel on Digital Cooperation, chaired by Jack Ma and Melinda Gates, makes five sets of recommendations: (i) build an inclusive digital economy and society; (ii) develop human and institutional capacity; (iii) protect human rights and human agency; (iv) promote digital trust, security, and stability; and (v) foster global digital cooperation.
- The twin goals are (i) end extreme poverty: reduce the percentage of people living on less than $1.25 a day to 3 percent by 2030; and (ii) promote shared prosperity: improve the living standards of the bottom 40 percent of the population in every country.
- Although the 2019 Development Committee mainstreaming paper used the term disruptive and transformative technologies (DTT) (which the Independent Evaluation Group [IEG] uses), the World Bank Group’s most recent mainstreaming paper uses the term DDT (digital and disruptive technologies) with no change in the definition.
- The scale of mobile money continues to grow, with more than 900 million accounts in 90 countries and $1.3 billion in daily transactions (Pasti 2018).
- For organizational effectiveness literature, see, for example, Burnard and Bhamra 2011; Christensen 1997; Christensen, Anthony, and Roth 2004; Christensen and Overdorf 2000; Christensen and Raynor 2003; Hatum and Pettigrew 2006; Sheffi 2007.
- For relevant IEG evaluations, see, for example, World Bank 2011a, 2012a, 2013, 2015a, 2016b
- The detailed list of preparedness dimensions based on the literature is (i) organizational ability: knowledge and thought leadership; staff skills; resources; organizational structure, roles, processes and decision-making that allow for flexibility; agility, knowledge sharing and learning, and cross-sectoral coordination and collaboration; ability to identify, adapt, and deploy knowledge on technological developments; and partnerships, global networks, and positioning relative to other actors; and (ii) organizational willingness: leadership, culture, and incentives for innovation, and risk appetite and informed risk taking.