The need for collaboration is underlined by the wide-ranging nature of disruptive and transformative technologies (DTT) projects that cut across sectors and often involve both public and private institutions.
This evaluation found that there are examples of effective collaboration for DTT support, for example Development Data Partnership (formerly Data Collaboratives), the Digital Central Asia–South Asia program, Geo-Enabling for Monitoring and Supervision, and Identification for Development.
However, overall, there was a perception among interviewees from across Global Practices that collaboration is insufficient among the different Global Practices, with instances of competition for task team leadership.
Given that contributions from both public and private sectors are necessary for successful DTT-related outcomes, the lack of collaboration between the World Bank and the International Finance Corporation was also an issue.
Procurement is a major implementation constraint in DTT-related projects.
There is only fragmented guidance for complex technology projects, specifically on the risks associated with these projects (such as vendor lock-in, lack of interoperability of systems, potential loss of data ownership and lack of data privacy, limitations in technology-specific legislation and regulation, and mismatch between business processes and the requirements of the particular technology). There is also insufficient guidance on how the flexibility available for complex technology projects in the World Bank’s procurement systems can be applied in different situations.
The need for greater collaboration in providing support in a cross-cutting area like DTT is underlined by the importance of the analog complements to digital technologies. These analog complements include improving the business climate, investing in people’s education and health, and promoting good governance (World Bank 2016e). Consequently, effectively harnessing DTT in development requires cross-sectoral links and collective (rather than individual) action. When an organization fails to foster sufficient collaboration among relevant departments in its DTT support, the division of labor may become inefficient and inhibit innovative solutions (Epstein 2019). The need for collaboration is also underlined by the role of both public and private institutions in implementing complex technology solutions (World Bank 2011a).
Bank Group DTT-related projects are wide ranging and illustrate why cross-sectoral collaboration is necessary. For example, a 2019 Tunisia project, GovTech Digital Transformation for User-Centric Public Services, aims to improve equitable access to and the quality and accountability of social protection and education services through a GovTech approach, targeting low-income groups, the vulnerable, women, the illiterate, and persons with disabilities. This wide-ranging objective would be addressed best by bringing to bear expertise in each of the identified areas (for example, collaboration across governance, education, poverty, social protection, digital development, competitiveness, and innovation). The importance of collaboration in providing DTT support—not just coordination or cooperation—is also emphasized in a recent publication: “Rather than the business specifying what was needed and telling the technology department (or vice versa), the two groups needed to think together about the issues, risks, and opportunities of the digital age” (Karacaoglu, Mocan, and Halsema 2018, 26).
Bank Group Recognition of Key Issues
- The Bank Group has emphasized the importance of collaborative work to solve development challenges in the face of emerging disruptions.
The Bank Group FY20–22 Human Resources Strategy recognizes the importance of catalyzing a culture of greater collaboration. It calls for the Bank Group to develop leaders who can mobilize resources across the Bank Group to solve increasingly complex development challenges. Furthermore, it emphasizes that the emerging disruptions require emerging leaders to be integrators and coalition builders and notes that “while technical expertise is critical, new leaders must go beyond individual excellence to contribute collaboratively to a shared purpose to help improve staff engagement and catalyze the WBG [World Bank Group] culture of greater collaboration and innovation” (World Bank 2019l, 12).
The Bank Group’s Mainstreaming paper recognizes the need to improve internal capabilities, including collaboration (World Bank 2019h). It notes that the Bank Group will ensure a coordinated, Bank Group–wide approach for DTT support on several fronts and support knowledge sharing and collaboration.
The Program Document for the World Bank’s 2020 MDDT Initiative refers to insufficient collaboration and coordination in mainstreaming digital and disruptive technology solutions across the Bank Group. High demand for greater collaboration and coordination was expressed in a series of consultations across the Bank Group. These noted that although there are tech initiatives managed by various units at the Bank Group, there is lack of cross-sectoral knowledge sharing and peer-to-peer learning (World Bank 2020d).
The World Bank is working to understand the state and nature of cross-sectoral collaboration. A forthcoming ASA, The Converging Technology Revolution and Human Capital: Potential and Implications for South Asia, reports that looking across the Human Development portfolio in all Regions, Human Development in South Asia stands out as the most “internally collaborative” (with joint projects between Education, Health, and Social Protection) (Bashir and others, forthcoming). However, it is also the least “externally collaborative” (with non–Human Development Global Practices), as measured by share of portfolio size either led by one Human Development Global Practice and contributed by others, or led by others and contributed by Human Development Global Practices. The level of collaboration with non–Human Development Global Practices in South Asia has been less than in most other Regions. Furthermore, the ASA points out that following the COVID-19 onset, pipeline projects under preparation indicate that South Asia Human Development has started to work with Global Practices focused on urban development, social development, and private sector development. But, paradoxically, there is very little collaboration with those Global Practices that could complement investments in technology that promote human capital (for example, Digital Development; Governance; Macroeconomics, Trade, and Investment; Energy; Water; and Agriculture).
- There have been efforts to improve collaboration within the World Bank, and there are some examples of effective collaboration in providing DTT support.
The Bank Group has attempted to improve collaboration for DTT support across Global Practices. For example, in July 2019, the Infrastructure Vice Presidency and the Equitable Growth, Finance, and Institutions Vice Presidency issued a joint memorandum agreeing on a mechanism whereby the Digital Development Global Practice would have coordination responsibility for all digital engagements with the country teams. However, the memorandum did not address leadership and coordination of DTT work, corporate responsibility for which currently lies with a small team housed in the front office of the Infrastructure Vice Presidency (see box 4.1). In July 2019, a new Disruptive Technology Task Force was set up under this team. The Disruptive Technology Task Force comprises one representative from each operational vice presidency and aims to follow up on the Mainstreaming paper by supporting the tracking of corporate priorities, new disruptive technology initiatives, knowledge sharing, and capacity building for staff and clients. New concerns that the internal realignment—which took effect on July 1, 2020, and involves the appointment of regional directors and maps technical staff to Regions—might hinder improved collaboration by recreating old problems of low mobility across Regions and lead to insufficient DTT specialized staff in each Region. The realignment’s effects on collaboration—in terms of quantity and quality—among the various parts of the World Bank will need to be monitored at three levels: (i) among Global Practices within a Practice Group; (ii) among Global Practices across Practice Groups; and (iii) among global directors, regional directors, and country directors.
To further foster DTT work, the 2020 MDDT Initiative proposes improved institutional arrangements. A Network and a Technical Working Group aimed at connecting teams and programs across the Bank Group that are relevant for DTT are to be established. In addition, the Bank Group proposes a Secretariat, housed in the Digital Development Global Practice, to support the Network and the Technical Working Group and implement an initial set of MDDT activities. As the initiative is rolled out and the detailed roles, responsibilities, and activities of the Secretariat in relation to other parts of the Bank Group engaged in DTT work are finalized, the following considerations can help avoid pitfalls and enhance the effectiveness of the Bank Group’s DTT work:
- Ensure that decision-making on DTT-relevant issues for sectors (such as agriculture, education, health, transport, or energy) and/or themes (such as data/development economics, or governance) continues to be made in the unit in which the expertise for that sector or theme lies.
- Closely monitor the extent to which the proposed institutional arrangement—Network, Technical Working Group, and Secretariat—is addressing the collaboration problem it was meant to solve.
- Ensure that DTT-relevant issues that cut across sectors, themes, or projects are addressed at the institutional level, rather than on an individual project-by-project basis.
- Ensure that the involvement of technology companies in the new institutional arrangement is appropriately ring-fenced to avoid compromising the Bank Group’s honest broker role and to ensure there is no conflict of interest with regard to, for example, procurement reform.
Box 4.1. Institutional Arrangements for Digital Technologies and DTT in the World Bank Group
In the World Bank
In 2020, as part of the Mainstreaming Digital and Disruptive Technologies (MDDT) Initiative, the World Bank Group proposed a Network, a Technical Working Group, and a Secretariat housed within the Digital Development Global Practice, to implement an initial set of MDDT activities.
Prior to the MDDT Initiative, the World Bank’s Digital Development Global Practice, housed in the Infrastructure Vice Presidency, had corporate responsibility for “digital engagements” within the World Bank. This Global Practice both provided task team leadership for projects and performed knowledge and coordination functions.
Corporate responsibility for disruptive and transformative technologies (DTT) rested with a small team located in the front office of the Infrastructure Vice Presidency. This group did not provide task team leadership for projects. It played knowledge and coordination roles, mostly through a Disruptive Technology Task Force, which was set up in July 2019 and comprised a representative from each of the operational vice presidencies.
Digital engagements and DTT also form part of the work of the Finance, Competitiveness, and Innovation and Governance Global Practices (and to a lesser degree, the Macroeconomics, Trade, and Investment Global Practice) under the Equitable Growth, Finance, and Institutions Vice Presidency and that of each of the sectoral Global Practices under the Social Development and Human Development Vice Presidencies. Each of these Global Practices provides task team leadership for projects in addition to playing knowledge and coordination roles in their respective sectors.
The Technology and Innovation Lab in Information and Technology Solutions provides support to operational teams engaging in technology-related piloting and innovation. Development Data Partnership (formerly Data Collaboratives), Disruptive Technologies for Development Trust Fund, Geo-Enabling for Monitoring and Supervision, Geospatial Operations Support Team, and Social Development Data Lab are among the other groups in the Bank Group that also deal with digital engagements and DTT and that support operational teams by playing a knowledge function. Staff managing these groups do not provide task team leadership for projects.
In the International Finance Corporation
The International Finance Corporation (IFC) seeks to support DTT through several core units described below. In addition, IFC’s mainstream industry departments and advisory functions support DTT interventions.
IFC restructured the DTT practices in fiscal year 2019. Prior to the reorganization, the Telecoms, Media, and Technology teams, fintech teams, and Private Equity and Venture Capital Funds teams were housed in a single department. Since the reorganization, the Telecoms, Media, and Technology group is part of the Infrastructure and Natural Resources Department; the fintech team has migrated to the Financial Institutions Group; and the Private Equity and Venture Capital Funds team has moved to a new Disruptive Technology and Funds group headed by a senior director. This new group aims to better integrate disruptive technologies across IFC through joint ventures and knowledge sharing with other departments.
In addition, the implementation of IFC’s Strategy 3.0 involved the creation of new IFC Upstream units in fiscal year 2020. Upstream units were created under each of the industry groups, including for infrastructure and disruptive technology, to support market creation initiatives and pipeline development for IFC. The Upstream units are also expected to be points of interaction with the World Bank.
Source: World Bank data and Independent Evaluation Group interviews.
Examples of effective cross–Global Practice collaboration for DTT for development exist. Interviewees pointed to ID4D, GEMS, the Digital CASA program, and the Development Data Partnership (formerly Data Collaboratives). Box 4.2 discusses the collaborative aspects of ID4D (box 2.1 provided ID4D lessons) and the Digital CASA program, and appendix D discusses GEMS and the Development Data Partnership.
In a recent first, the Vietnam country team collaborated with the Bank Group’s corporate disruptive technology team (in the front office of the Infrastructure Vice Presidency) to review ongoing and pipeline projects for DTT opportunities. Ongoing and pipeline projects in the Vietnam country portfolio were examined to check whether technology was already being deployed and whether there were new opportunities to do so. Among ongoing projects, the review identified 35 projects where DTT-relevant opportunities could potentially be harnessed. These 35 projects were in the following sectors: education; energy and extractives; environment and natural resources; health, nutrition, and population; social protection and jobs; transport; and water. The next step would be to seize missed opportunities, and the review has identified specific operations for project-level deep dives. Furthermore, the corporate disruptive technology team has developed a general framework for examining the opportunities and risks of technology adoption in infrastructure operations—a framework that could be examined for possible application to other sectors as well.
Box 4.2. Cross-Sectoral Collaboration in the ID4D and Digital CASA Programs
The Identification for Development (ID4D) program, launched in 2014, reports to a senior directors’ group, which includes 10 members who meet regularly to ensure a cohesive and coordinated approach across different sector entry points (for example, financial inclusion, social protection, and gender). The ID4D initiative, led by a program manager and a core team of six full-time staff members, works closely with an extended team of World Bank Group colleagues who are part of a formal cross-sectoral working group, consisting of lead specialists from each of the Global Practices and departments involved (Digital Development; Jobs and Social Protection; Health, Nutrition, and Population; Finance, Competitiveness, and Innovation; Governance; Gender; and Legal), and the Disruptive Technologies and Funds industry group of the International Finance Corporation (IFC). These staff provide a significant focus on ID4D and ensure links to the operational work of the Bank Group. ID4D staff provide critical advice and knowledge to operational teams but are not task team leaders of operations. The ID4D work program was developed with input from sector experts, including the leads from the five Global Practices, the Gender Global Theme group, the Consultative Group to Assist the Poor, and IFC. The initiative includes three pillars: World Bank operations/country engagement, thought leadership and analytics, and global platforms and convening. A ID4D high-level advisory council chaired by a World Bank managing director and 11 global thought leaders from the private sector and government meets twice a year with the aim of supporting global advocacy and providing a sounding board on emerging issues. In addition, partnerships with donors, the United Nations, and private sector and development partners help leverage one another’s strengths and achieve alignment on good practices in ID4D.
The Digital Central Asia–South Asia (CASA) program shows the importance of Country Management Units in eliciting multisectoral collaboration within the sectoral governance structure of the World Bank. Within this context, the Country Management Units need to play an essential role to create adequate room and resources for the appropriate integration of digital technologies in the country program. Such Country Management Unit leadership was a key enabler in facilitating the sample of projects in Kosovo and Nicaragua reviewed for this evaluation. Conversely, in another case, the collaborative nature of the project was not sufficiently appreciated and the budget was assigned to one Global Practice, even as key inputs were expected from lead specialists in three different Global Practices; the budget was depleted quickly with no genuine collaboration.
Instead of relying on the usual work program agreement process, which tends to discourage collaboration as teams compete for budgets to cover their own fixed costs, in the Afghanistan Digital CASA project, the practice managers worked together to prepare a team-based budget, distributing the budget according to tasks and costs. Then they compared this distribution against the work program agreement budget to make sure the teams were covered. This allowed the teams to finance a geospatial data specialist to work with several Global Practices and for the Finance, Competitiveness, and Innovation and the Social Protection teams to work together.
The Digital CASA projects in Afghanistan and the Kyrgyz Republic combined public and private sector engagement. The Digital Development Global Practice served as the repository of technological knowledge. The Finance, Competitiveness, and Innovation and the Governance Global Practices provided their experience in the applications of technology, and IFC’s Telecoms, Media, and Technology group contributed its experience with information technology and telecoms in the private sector.
Source: Document reviews and Independent Evaluation Group interviews.
- Bank Group databases lack sufficient data on collaboration relating to DTT.
Adequate data on internal Bank Group collaboration relating to DTT do not exist, despite the cross-cutting nature of DTT. Existing Bank Group data on cross-support among Global Practices from Bank Group databases, or on collaboration among Global Practices from the Bank Group’s engagement surveys, fall short for the following reasons: (i) there is no official Bank Group list that identifies DTT-related projects and ASA (except for Digital Development or ICT projects and ASA); (ii) disaggregated data on cross-support by Global Practices from Bank Group databases do not provide information on the extent of collaboration specifically in DTT-related projects and ASA; (iii) although the Bank Group’s engagement surveys do capture aspects of collaboration, disaggregations specifically for DTT-related projects and ASA are unavailable; and (iv) in any case, none of the above data say anything about the nature and quality of the collaboration in enhancing the outcomes of DTT-related projects and ASA.
- In interviews, Bank Group staff reported that collaboration across Global Practices has been insufficient in scale and scope.
IEG interviews conducted for this evaluation pointed to insufficient DTT-relevant collaboration among the different Global Practices. NVivo analysis of IEG’s interviews of Bank Group staff that are part of the reference population knowledgeable about collaboration and coordination (described in appendix A) found that of the 46 interviewees, 37 (or 80 percent) reported insufficient collaboration in providing DTT support. Of the 29 interviewees who talked about internal structures and collaboration, 23 interviewees (or 79 percent) reported that these had been inadequate to support collaboration. There was a perception among interviewees from across Global Practices that collaboration is insufficient among the different Global Practices, with instances of competition for task team leaderships. Interviewees further observed that ICT may be embedded in many Bank Group projects in a country, but each project team often has its own IT expert; each expert could give different advice regarding data collection and management. The result is that the systems “do not talk to each other.” Interviewees noted that many issues related to budget allocation and recognition arose when two or more Global Practices or vice presidencies were involved in a multisectoral project. Moreover, even in digital economy diagnostics, which were intended to encourage collaboration across Global Practices, collaboration problems persist (Hanna 2019).
Previous IEG evaluations have flagged general collaboration challenges within the Bank Group. IEG’s Knowledge Flow and Collaboration evaluation pointed to a lack of collaboration for integrated solutions at the Bank Group (World Bank 2019g). Furthermore, IEG’s evaluation Support for Innovation and Entrepreneurship found that the flow of knowledge on innovation and entrepreneurship is limited, especially across networks, sectors, Regions, and the three Bank Group institutions (World Bank 2013).
The World Bank and IFC could play separate but complementary and highly coordinated roles in DTT projects. IFC staff highlighted the importance of the World Bank in persuading governments to implement regulatory changes, while the World Bank needs IFC’s investments to incentivize reform on the part of governments. This collaboration works either when the incentives of the two institutions are aligned or in an ad hoc manner largely based on personal relationships among IFC and World Bank staff. The lack of more systematic collaboration is partly due to the discrepancy in the timescales of the two institutions: the regulatory reforms implemented by the World Bank often span a much longer time than IFC’s investment projects. In such cases, the two institutions can have separate but complementary roles rather than committing to joint operations. Even with separate roles, ensuring complementarity requires collaboration. This evaluation found that collaboration works when IFC and the World Bank engage upstream even before any lending or investment projects are in the pipeline. Such preengagement was a major factor contributing to the success of the Digital CASA program. Conversely, in DE4A Moonshot/Accelerate, IFC was brought in late in the process, which initially created hurdles, although the teams were eventually able to overcome these. A One World Bank Group approach would also help with funding and data issues relating to areas where IFC is not currently involved (for example, Digital Economy Sandbox funding). Internally in IFC, collaboration remains a work in progress since IFC’s realignment of the Global and Regional Mainstream units and the addition of the Upstream units.
Although the DE4A country diagnostics are an example of different Global Practices coming together for a more coherent approach to DTT, they illustrate some shortcomings in collaboration. DE4A country diagnostics mainly involve the Digital Development, the Finance, Competitiveness, and Innovation, and the Governance Global Practices. At the review meeting for the initiative, the need for engaging with additional Global Practices such as Macroeconomics, Trade, and Investment and Poverty was noted, but no subsequent action was taken (Hanna 2019, 70).
The lack of systematic collaboration across Global Practices remains an issue for several reasons. The reasons include (i) linking budgets with task team leadership, often resulting in Global Practices lobbying country directors to secure task team leaderships; (ii) holding regional directors and practice managers accountable for ensuring that each of their staff is fully employed and requiring that preset lending volume targets are met; (iii) giving most, if not all, recognition to the task team leader rather than also recognizing co-task team leaders and other team members; (iv) having unclear division of roles and responsibilities for digital technologies and DTT (see box 4.3 for details); and (v) importantly, having a lack of emphasis on the achievement of intended development outcomes.
Box 4.3. Division of Roles and Responsibilities for Digital Technologies and DTT
Staff interviewed for this evaluation pointed to a lack of clarity with regard to the following issues which, according to them, resulted in unreasonable transaction costs and hampered the efficient delivery of lending and nonlending operations:
- Although the coordinating role of the Digital Development Global Practice was clear with regard to information and communication technology and digital technologies, there was ambiguity about the role of this Global Practice for coordination and task team leadership when it came to sectoral operations, such as digital agriculture, energy, transport, water, edtech, and e-health.
- World Bank administrative resources and staff time was taken up with competition for task team leaderships among the Digital Development; Finance, Competitiveness, and Innovation; and Governance Global Practices. Lack of collaboration among different experts ran the risk of inconsistent advice on the same issue by different parts of the Bank Group (such that the systems “do not talk to each other”).
- There was insufficient clarity about the role of the Digital Development Global Practice when it came to operations involving disruptive and transformative technologies (DTT) (beyond information and communication technology and digital technologies) such as robotics, artificial intelligence, internet of things, nanotechnology and biotechnology, batteries, drones, solar panels, and self-driving vehicles.
- Interviewees further noted that there was fragmentation of DTT work within the Regions, with the risk of missed opportunities for lesson learning. They suggested regional focal points—similar to gender focal points—as a possible way to help coordinate DTT work within a Region.
Source: Independent Evaluation Group interviews.
Procurement (World Bank Only)
In DTT-related projects, procurement can raise specific concerns due to (i) an incomplete understanding (by decision makers, client IT specialists, and World Bank staff) of functional and technical specifications relating to interfaces, protocols, and data structures; (ii) consequent potential difficulties in interoperability across systems and the potential for continued dependence on specific vendors; (iii) potential need for business process reengineering relating to, for example, the conversion from analog to digital data; (iv) potential loss of data ownership and privacy and the commoditization of user data by vendors; and (v) limitations in technology-specific legislation and regulation.
Bank Group Recognition of Key Issues
- The World Bank has recognized that procurement is a significant constraint to innovation within the World Bank and that there is also need for continued innovation in DTT-relevant procurement.
According to the Mainstreaming paper, “procurement can play a strategic role in enabling innovation and mainstreaming disruptive technology into Bank operations” (World Bank 2019h, 16). The paper notes the need for continued innovation in DTT-relevant procurement. A 2017 World Bank report found that staff considered procurement to be a significant constraint to innovation within the World Bank, due to long delays that affect the flexibility needed for innovation, and, in particular, difficulties in hiring young firms (World Bank 2017c). These issues are especially detrimental to DTT-related projects.
- Procurement is a major implementation constraint in DTT-related projects in the eyes of most World Bank interviewees.
Staff interviewed by IEG overwhelmingly reported that the World Bank’s procurement systems are not supportive of DTT and constrain the institution’s ability to respond to client needs. NVivo analysis of staff interviews conducted by IEG showed that of the 24 interviewees who talked about procurement systems, 21 interviewees (or 88 percent) reported a lack of DTT-relevant procurement systems conducive to DTT-relevant work. The NVivo analysis is described in appendix A. Interviewees reported that procurement processes do not sufficiently encourage cases where the World Bank or borrower may be unaware up-front of the best solutions available in the market, which is frequently the case with innovative DTT-related projects. For example, regarding procuring services related to AI, clients and the World Bank may not have a priori specifications but may have knowledge only of the problem to be solved. Ideally, vendors would submit their best solutions based on the problem to be solved.
World Bank staff interviewees pointed to trade-offs between helping clients to customize specifications to adapt to present needs and maintaining interoperability with both legacy and future systems. Interviewees pointed to capacity limitations, not only at the level of the borrower but also at the level of World Bank operational and procurement staff, flagging the need to develop ICT procurement skills, encourage principles of innovation, and streamline processes. Because clients typically need software and an after-sale relationship with the vendor, particular skill is needed to navigate the fine line between customizing specifications on behalf of the client and drawing up specifications that are too generic to be useful. If specifications are too customized, the outcome will tend toward single sourcing, which would put the World Bank in the position of advocating for a single supplier.
Several past IEG evaluations have flagged procurement issues in technology projects. For example, IEG’s evaluation Capturing Technology for Development reported that ICT procurement was a major constraint for implementation due to the lack of expertise in ICT procurement, the need to separate the procurement of goods and services, inconsistent advice to borrowers, limited borrower capacity, and the lengthy procurement process (World Bank 2011a). Management accepted that procurement was an issue, and procurement guidelines were revised in 2011 to include ICT-specific approaches, procurement methods that reflected the complexities of ICT system design and development, and the review of bidding documents for ICT projects and guidelines for task team leaders. IEG’s evaluation The World Bank Group and Public Procurement confirmed a series of improvements to ICT procurement, including revisions to the standard bidding documents, acceptance of two-stage bidding, the use of framework agreements for software maintenance or servicing, and other means of flexibility, although legacy difficulties remain (World Bank 2014b). In the following year, IEG’s second evaluation on learning and results noted that World Bank staff sometimes lacked the knowledge needed to check whether procurement specifications for high-technology data systems were adequate, making them vulnerable to companies seeking to sell expensive systems that exceeded client needs (World Bank 2015a).
- IEG’s expert review undertaken for this evaluation found that there is currently only fragmented guidance for complex technology projects, specifically on the risks associated with complex technology projects (such as vendor lock-in, lack of interoperability of systems, potential loss of data ownership and lack of data privacy, limitations in technology-specific legislation and regulation, and mismatch between business processes and the requirements of the particular technology).
IEG’s expert review of the World Bank’s procurement regulations from the perspective of DTT-related operations found that World Bank procurement regulations and guidance do acknowledge a range of DTT-relevant factors. These include (i) the need for procurement to consider the speed of technological change; (ii) the need for transferability and security of information across past and future systems and across different providers; (iii) the dangers of continued dependency on particular bidders; and (iv) possible constraints on eligible technical solutions due to limited internet access or mobile bandwidth and coverage. The review also referred to the importance of specifying functional over technical requirements and allowing borrowers a wide range of market approaches to procurement, including of new technologies. Although offering a range of selection options and market approaches to procurement, the World Bank’s procurement regulations and guidance only offer a generic guide to risk management, which is specific to neither IT nor DTT.
Current procurement guidance has limited relevance for risk factors that greatly influence the success or failure of complex technology projects. These factors include the following:
- Borrower capacity issues, including technology staff capacity
- Functional and technical specifications that may not be well understood by decision makers or developers
- Interoperability, not only with legacy systems but also with potential future directions
- Business process reengineering, for example converting from analog to digital systems and processes
- Uncertain costs arising from unclear buyer or supplier transparency of responsibilities or ongoing license or maintenance costs, or both
- Legal challenges (box 4.4)
Procurement requirements for technology are partly dealt with in the standard procurement documents, but critical aspects are only briefly mentioned, such as process reengineering, the significance of open standards, and systems piloting.
The World Bank does permit the use of two-stage bidding for complex ICT and DTT procurement, for example in cases where the exact solutions are not known up-front. In such cases, unpriced technical proposals are invited, based on performance specifications or on design concepts. Bidding documents may be adjusted based on information acquired during the process. Even this approach may not be sufficiently open ended for certain areas of DTT, such as AI, where it may be possible to specify only the problem to be solved. Electronic-reverse options are another flexibility that has been introduced. In principle, the World Bank also offers a “competitive dialogue” process that allows for wide flexibility.
Box 4.4. Legal Challenges in Procurement
One example of legal issues arising in complex technology was the failure to specify ownership and usage rights for an application programming interface in Estonia, leading to litigation years later between the government and the private vendor.
Interviewees pointed out that copyright of applications purchased from the private sector can also lead to downstream legal issues. Moreover, as data become digital, the distinction between content and data blurs, leading to issues of ownership, privacy, and data use.
Country clients also mention possible cross-border legal issues, for example those related to cloud storage of information in servers in other countries or with certain large-scale private data companies. Cloud storage in the United States or elsewhere could be problematic if, for example, political sanctions are imposed that deny countries access to their data or compromise privacy. Storage with some private providers could open the door to questions about the ownership or confidentiality of information. One solution could be that the World Bank advises clients on how to classify their data. Risk-free data (such as that collected from the web) could be given to private company servers, while sensitive information, such as health or security-related information, could be hosted on in-country servers. But data sovereignty issues will need to be resolved.
Source: Independent Evaluation Group interviews for the expert review undertaken for this evaluation.
In addition, the World Bank’s procurement systems provide some flexibility in the thresholds for bidders’ years of experience. This facilitates bidding by relatively young start-ups but may still deny entry to the youngest firms, which could be the most innovative. However, thresholds are needed to safeguard the buyer.
Task team leaders are reluctant to use the available flexibility as this is perceived to add to the time and complexity of an operation. Interviewees attribute the delays and complexity to insufficient guidance on how to apply the available flexibility for complex technology projects in the World Bank’s procurement systems in different situations. Providing guidance can encourage project teams to use the available flexibility for complex technology projects. This will require that procurement staff are well versed in the guidance to help ensure that projects proceed smoothly and without undue delay. Reluctance to use the available flexibility may also result from misaligned incentives that encourage risk aversion.
- The six Global Practices would be Governance, Jobs and Social Protection, Education, Poverty, Digital Development, and Finance, Competitiveness, and Innovation. In practice, the Poverty and Finance, Competitiveness, and Innovation Global Practices are not involved in the project at this time.
- Normally, the country director only gives the budget to the Global Practices it chooses for task team leadership. If a Global Practice is not providing task team leadership for a project, it cannot show in the system that it is working on a project (it is only shown as “cross-support”).
- Hanna (2019) noted that in digital economy diagnostics, a preoccupation with capturing budget has come at the cost of sufficient collaboration, a lack of analytical depth, insufficient attention to the twin goals, and a failure to connect to the analog foundations stressed by the World Development Report 2016 (World Bank 2016e). He concluded, “The underlying factors for these gaps go beyond having common assessment tools. The current operating model tends to inhibit collaboration and cause fragmentation, internal competition.…These gaps are not insurmountable, within the World Bank and in country clients and there are a few encouraging examples. But overall assessment of all pilots suggests there is much room for improvement” (Hanna 2019).
- Staff pointed out that the World Bank could better use its convening power to identify specific country risks for different stakeholders.
- Technical specifications relate to interfaces, protocols, data structures, system language, and standards that ensure interoperability across vendors, legacy systems, programming languages, data formats, and the like. Efforts should be made to adopt or specify open standards that are approved by recognized standards organizations and available for public use free of royalties. These can help eliminate the problem of being tied to a specific vendor.
- Converting analog to digital information to enable numeric analysis will commonly require a review and, often, the overhaul of all systems, processes, methods, rules and regulations, and legislation. Stakeholders often resist this. World Bank staff said that the digitization rate is as low as 30 percent in some client countries where most financial systems are manual and point out that the World Bank can introduce clients to a variety of alternative digitization tools.
- Two draft World Bank analytical reports—GovTech Procurement Strategy and Disruptive Technologies in Public Procurement—are in preparation.