The vision for the World Bank’s 2016 procurement reform was to help clients achieve better value for money with integrity in delivering sustainable development. Procurement in World Bank–supported investment project financing (IPF) activities was valued at nearly $20 billion from fiscal year (FY)17–23. The World Bank’s procurement processes need to ensure that clients get the best value for every development dollar. The reform was driven by procurement international good practices and the need to address major procurement bottlenecks in project implementation. The reform introduced a new procurement policy, directive, and procedures. The Governance Global Practice and then, from FY21, the regions, were responsible for overseeing procurement implementation, with the Equitable Growth, Finance, and Institutions Global Practice Group reporting on implementation progress and Operations Policy and Country Services leading the procurement policy aspects of the reform.

The reform also aimed to help clients better support projects’ needs and achieve results through outcome orientation and data analytics. The reform emphasized seven core principles to achieve this objective: value for money, the central principle to be achieved through greater efficiency, economy, integrity, fairness, transparency, and fit for purpose. The reform introduced a procurement strategy tool at the project level, simplified procurement review process, enhanced risk assessment for decisions on procurement, and highlighted approaches to support quality and sustainability. It also focused on capacity strengthening to help clients implement procurement in accordance with the seven principles.

This evaluation assesses procurement in World Bank–supported IPF and related capacity strengthening since the 2016 procurement reform. The findings produce lessons and recommendations to help improve the ongoing implementation of the reform. The report focuses on progress in advancing the seven core principles and efforts to strengthen client capacity in these areas. The evaluation analyzes procurement data from 713 IPF projects in 112 countries and over 900 capacity strengthening activities and projects. The evaluation draws on a literature review, over 70 case studies of project procurement, over 200 key informant interviews, analysis of external procurement data to estimate the procurement capacity of countries, and two surveys of over 200 World Bank staff (one for procurement staff and one for task team leaders).

Overall, the evaluation finds that the reform’s sound logic has led to promising results, but implementation needs to be scaled up. The reform is a major step toward making procurement more adaptive, faster, and more transparent. It enhances integrity, fairness, and transparency. When projects apply the procurement principles, they increase value and achieve better implementation and procurement performance. However, the reform is still under implementation and has not broadly achieved the seven principles. Moreover, client capacity strengthening needs strategic attention to increase the scale of reform elements in countries and support change management processes for such a substantial reform.

Further progress requires the World Bank to enhance the reform’s change management. This would require the World Bank’s management to centrally oversee implementation of the reform, routinely monitor outcomes toward the procurement principles, and strategically optimize the allocation of procurement staff and other client support to address project- and country-level needs based on risks and capacity constraints. Progress requires a change management process that involves (i) World Bank procurement staff coaching clients and task teams in World Bank–supported projects to strategically plan, customize the design of procurement, and analyze markets; (ii) increasing incentives and collaboration between technical and procurement teams within the World Bank to help clients optimize quality and sustainability approaches; (iii) strengthening procurement post reviews and risk assessments; (iv) rethinking procurement capacity strengthening to move toward the risk-based use of client country systems in IPF; (v) conducting dialogue and capacity strengthening at the country level to address common efficiency bottlenecks in projects; and (vi) leveraging data analytics to bolster implementation of the reform across regions and allocate staff support for where and when it is most needed.

Advances in Procurement Efficiency

Procurement processing times have quickened since the reform, although the efficiency of more complex and competitive procurements could be enhanced. Procurement efficiency refers to the speed and ease of procurement. Since the reform, average procurement processing times have been as much as three times faster compared with the 2014 Independent Evaluation Group procurement evaluation. This quicker pace is largely a result of the World Bank raising the value threshold for carrying out a prior review (comprehensive and time-consuming review of the client procurement activities during procurement implementation) and instead carrying out a post review (review of procurement activities on a sample basis after the client completes the procurement activity). Procurement processing times are also faster when simple procurement approaches are used. However, competitive, high-cost, high-risk, technically complex procurements and procurements that focus on quality and innovation still have long processing timelines, similar to before the reform. Although longer procurement times would be expected in these types of procurements, finding ways to simplify processes could further enhance efficiency and project implementation.

Efficiency is reduced when procurements start late or encounter processing issues, and when clients have low procurement capacity. Signing key contracts early in project implementation appears important. Procurement issues—when problems arise while processing procurement activities—frequently occur in competitive procurements when clients prepare technical documents, wait for bids for works, evaluate bids and proposals, and prepare for contract signatures. Not addressing these issues as they arise compounds them over time and further delays project implementation. Efficiency also drops when clients’ procurement staff lack experience applying the procurement approaches planned in World Bank–supported projects.

The reform’s launch of an electronic procurement system helps World Bank procurement staff review documents, but making the system easier for clients would enhance efficiency and data capabilities. The Systematic Tracking of Exchanges in Procurement (STEP) increases the convenience of tracking procurement activities in one place, facilitates exchanges between clients and World Bank staff in World Bank–supported projects, and reduces World Bank staff’s response time to client requests. However, clients do not find STEP user-friendly and report that entering activities and uploading documents to STEP is arduous and time-consuming, especially for simplified post review processes that enable clients to conduct procurements on their own. Low-resource contexts heighten the STEP burden, when data entry can take days. World Bank staff spend considerable time helping clients with STEP, and delays occur in uploading information, limiting timely data analyses and oversight. Moreover, when countries have their own procurement tracking systems, STEP duplicates data entry efforts.

Procurements conducted by clients on their own using simplified post review processes contribute to timeline acceleration and client empowerment but limit formal World Bank staff–client learning. Clients appreciate the reform’s emphasis on simplified post review procurement because it empowers them to conduct their own processes. However, clients benefit from learning how to conduct procurement using World Bank standards and approaches. This learning happens when clients and World Bank procurement staff engage to review procurement processes and identify real-time issues for improvement. Emphasizing simplified post review procurement reduced formal opportunities for knowledge transfer and learning and left clients reliant on informal and often ad hoc support from procurement staff. For higher-risk procurements or procurement stages with reoccurring processing issues, clients often want closer World Bank oversight to ensure proper processing. For example, World Bank oversight could be more targeted and directed toward the specific procurement stages that are affected more often by processing issues, and this, in turn, could raise clients’ confidence.

Advances in Procurement Economy, Integrity, Transparency, and Fairness

The 2016 reform emphasized enhancing procurement economy, but the practical uptake of this principle has been slow. Economy refers to cost and noncost factors that contribute to procurement value (such as competitiveness, quality, and sustainability) and social, economic, and environmental benefits. The reform introduced new approaches to increase the consideration of cost and noncost factors in procurement, but few projects use them. Strengthening procurement economy entails leveraging market analyses to identify qualified suppliers and bolster competition, emphasizing quality throughout the procurement cycle, and adopting procurement approaches that pursue quality and sustainability and not just the lowest cost.

Procurement is still not as competitive as the reform envisaged, and clients often struggle to identify qualified suppliers, especially without proper market analyses. Quality suppliers are often hard to find in small countries, weaker economies, or fragile and conflict-affected situations. This contributes to about half of procurements in World Bank–supported projects using limited and direct selection approaches (which can ease and speed up contracting in underdeveloped or challenging markets but makes the process less competitive, transparent, and cost effective). Clients’ project procurement strategy tool includes market analyses to identify potential qualified suppliers for procurement activities. However, these analyses are often carried out too early and superficially to produce actionable information. Examining common market needs across projects in regions could be an opportunity to simplify the market analysis process. The evaluation finds that practical market engagement and analysis tools, such as supplier databases or workshops to communicate opportunities to qualified suppliers early in project implementation, could make procurement more successful.

The reform introduced approaches to improve procurement quality and outcomes, but project teams continue to use familiar approaches they know best. The evaluation finds that projects rarely use these new approaches to improve quality, competition, and sustainability and to customize procurement. For example, negotiations and rated criteria to improve quality of works are seldom used. Clients rely on familiar approaches because they often do not understand how to use the new ones, which rarely are part of their established practices. Moreover, World Bank procurement staff often do not engage in country-level dialogue with clients to develop these new practices. World Bank staff also lack the incentives or knowledge to help clients apply more appropriate and innovative approaches, which require new learning to implement.

Several projects use alternative procurement arrangements that follow development partners’ processes, but none follow clients’ national arrangements. The 2016 reform introduced alternative procurement arrangements to facilitate the implementation of projects jointly financed by the World Bank and other development partners. Consequently, clients can use the procurement arrangements of another partner instead of those of the World Bank and can also apply the procurement arrangements of their own agencies and entities to projects. In jointly financed projects, the World Bank has successfully applied the procurement arrangements of other partners. However, the analyses suggest that clients’ procurement arrangements have never been used because of the risks related to relying on them, even if their use in high-capacity, low-risk contexts might speed up procurement, save client and World Bank resources, and improve clients’ perception of the World Bank’s procurement framework.

STEP improves integrity and transparency in procurements, but technology could be better leveraged to enhance oversight and monitor procurement. Integrity refers to using resources according to their intended purposes and aligning them with the public interest. Transparency entails the review of procurement activities supported by adequate documentation and disclosure. Fairness represents equal opportunities for bidders and credible mechanisms to address procurement-related disputes and complaints. STEP increases integrity and transparency by tracking procurement activities and making this information available to clients and, partially, to the public. However, the World Bank could publish more data on projects’ procurement outcomes to better manage results and help clients use technology to monitor whether procurement activities reach beneficiaries, especially for contracts in more remote areas and in countries with high integrity risks. Moreover, STEP is not integrated with the World Bank’s financial management system, making contract management more difficult.

Reviews to audit simplified post review procurement done by clients could be augmented to ensure timely feedback for projects, especially in low-capacity contexts. Few projects have a procurement post review report during the first years of project implementation. This causes problems in course corrections, especially for clients in lower-capacity procurement contexts who need early feedback on their procurements to make adjustments and avoid repeat mistakes that could strain the World Bank–client relationship. World Bank procurement staff also apply different criteria to sample procurement activities for post review reporting, resulting in projects having uneven oversight attention. The guidance on post review reporting is vague, which allows too much room for variation in oversight practice and does not adequately consider the needs of clients with different capacities and risks. Moreover, there is a need for simplified data transfer from clients to ensure timely access to complete procurement data for the World Bank or a third party to conduct post review reports or audits.

The standstill period introduced by the reform helps enhance fairness; however, complaint handling could be quicker, and data could be better used to inform quality improvements. The standstill period provides a window during which the procurement processes are paused; thus, suppliers can submit complaints to clients, if applicable. This period helps improve fairness, but the quick handling of complaints poses an ongoing challenge for clients in World Bank–supported projects. In addition, the World Bank could better use data on complaints to improve its procurement support for projects and help clients enhance the quality of procurement practices. Often, complaints related to the quality of procurement documents, such as unclear descriptions of evaluation criteria, could be better addressed.

Advances in Procurement Fit for Purpose and Value for Money

The reform’s expansion of fit-for-purpose procurement activities is successful when project teams have the knowledge or support to implement them. “Fit for purpose” is the extent to which clients customize procurement approaches to the country context and project needs to create value for money; in other words, fit for purpose finds the “right solution” to specific procurement needs. The reform’s flexibilities, such as the menu of procurement approaches, are meant to help clients do just that. However, clients and World Bank staff often lack the knowledge or incentives to apply unfamiliar approaches.

World Bank procurement achieves value for money when it applies the reform’s principles. Value for money refers to receiving the greatest benefits from the price paid. Achieving value for money requires maximizing a procurement’s financial and nonfinancial benefits while addressing risks and applying the procurement principles as appropriate. The evaluation shows that when clients apply the principles, projects achieve higher implementation and procurement performance ratings, thereby achieving higher value for money. However, certain Regions, such as Africa and South Asia, achieve lower value for money; hence, barriers limiting value for money (such as long timelines for competitive procurement activities) could be better targeted. Furthermore, procurement performance ratings correlate with estimates of value for money and project implementation ratings.

Projects that process procurement activities early during their implementation tend to have better performance and value for money. Conversely, delays in starting procurement activities, often caused by the unavailability of experienced client procurement specialists, slow project implementation and jeopardize value for money. Projects supported by experienced procurement specialists have earlier procurement implementation, fewer procurement issues, and better performance regardless of country capacity.

Collaboratively prepared procurement strategies and data analytics could improve procurement-related decision-making. Even though the World Bank and clients recognize that strategic planning in procurement is important, it is not yet working well enough to help project implementation. The project procurement strategy for development, which the 2016 reform introduced to facilitate strategic procurement planning, is typically a generic document with similar content across many projects. Clients mostly prepare it to comply with World Bank requirements and rarely use it for procurement implementation. When strategic procurement planning works, it results from a collaborative effort between the World Bank’s and the client’s technical and procurement teams. The World Bank’s introduction of STEP and related data dashboards has advanced data analytics. However, these tools could still be improved to make the data actionable for decision-making by World Bank staff and clients and to increase value for money. Dashboards mostly report on efficiency indicators but without benchmarks or targets to assess progress, and they do not report other potentially relevant indicators on all procurement principles. As a result, clients and World Bank staff and managers rarely use these analytics to inform decision-making.

Procurement risk assessments could be improved by reducing their frequency, using data mining, thoroughly considering the full spectrum of procurement risks, and tracking mitigation actions. If the procurement activities necessary to support projects are not completed on time, they present a risk of reducing the project’s development outcomes. In practice, World Bank procurement specialists carry out risk analysis with little or no involvement of task team leaders and clients and rarely consider this type of risk in procurement. The spectrum of procurement risks to consider include those related to processing issues, integrity of post review processes carried out by the client, innovation, complexity, and client capacity, among others. Although the 2016 reform refers to many of these risks, their consideration at the project and activity levels is often not comprehensive and mitigation actions are weak. Risk analyses also focus on repeat assessments by World Bank procurement staff at the project level, which consume much time, do not significantly change during project implementation, and do not inform actions by clients. More emphasis could be placed on active monitoring and client dialogues on mitigation actions to learn what works to enhance procurement, mining data in the portfolio to identify issues that could provide real-time information to flag problems, and using data on individual activity risks.

Advances in Procurement Capacity Strengthening and Support

Since the 2016 reform, the World Bank has increased its procurement capacity strengthening, but few efforts focus on sustained change and countries with low capacity. Most of the World Bank’s capacity strengthening support since the reform is in the form of advisory services and analytics financed by trust funds, which increased by about 50 percent from FY10–16 to FY17–23. Capacity strengthening often focuses on activities such as short-term training of clients to increase procurement knowledge, skills, and awareness and the design of e-procurement systems. These activities are often fragmented and are not part of a coherent change process, and the results are not well measured. Some lending projects also support procurement capacity strengthening, but only about half of the indicators measuring results were achieved. Moreover, capacity strengthening activities less frequently target countries with lower procurement capacity, especially in Africa. The reform introduced hands-on expanded implementation support (HEIS), which has successfully helped low-capacity clients implement procurement in projects. HEIS, however, is mostly designed to alleviate clients’ procurement burden and not to systematically reinforce local human resource capacity.

The World Bank’s capacity strengthening support could be more strategic and target the most pressing needs in the portfolio. Only about 20 percent of World Bank Country Partnership Frameworks include activities that enhance country procurement capacity and indicators to measure procurement principles. Moreover, capacity strengthening activities are often disconnected from procurement bottlenecks critical to resolving to increase results. For example, recurring procurement issues encountered in many countries contribute to delays and jeopardize achievements in World Bank–supported project portfolios. To measurably improve project performance, there could be more efforts to target bottlenecks strategically and systematically so that they do not reoccur repeatedly—for example, lengthy bid evaluation stages, quality challenges in procurement design, and slow complaint handling.

Procurement capacity strengthening support by World Bank staff could be better guided, facilitated, and recognized. Much of the day-to-day client support by procurement staff is informal, untracked, or unmeasured and is often inadequately recognized by management. Clients, however, deeply appreciate procurement staff support when it is available. Procurement staff time is frequently spread over 10 projects, with greater dispersion in low-capacity contexts. World Bank procurement staff are encouraged to concentrate their efforts on comprehensive prior reviews and problem-solving in procurement that has a high monetary value. However, many procurement staff show interest in doing more to strengthen client procurement capacity, understanding its importance for the overall portfolio’s performance. Procurement staff could be better guided by strategic priorities for capacity strengthening in countries and benefit from coaching, tools, and access to resources to help them support clients in areas such as quality, data systems, complaint handling, and market engagement. The experience of procurement staff in these areas is often limited, and task team leaders and clients have even less experience.

Procurement-related training, HEIS, and knowledge sharing could be leveraged to address constraints on procurement human resource capacity, affecting the portfolio’s efficiency. Most World Bank support is geared toward short-term learning activities in projects that do not typically build sustained human capacity at the country level. The evaluation consistently finds that the underlying problem affecting project procurement performance is limited access to experienced human resources in countries. Frequent turnover of clients’ procurement specialists also undermines project implementation. HEIS fills a project-level human resource gap but, in its current design, does not consistently develop local experience, skills, or capacity to implement elements of the procurement reform. Creating platforms for peer-to-peer knowledge transfer among staff and clients could improve the experience in implementing quality and sustainable procurement approaches.

Recommendations

Recommendation 1. Improve change management support for the reform’s implementation.

Proposed actions are as follows:

• Ensure strong central oversight and governance arrangements to manage the reform’s implementation across regions. This support may involve proactive senior management leadership and incentives that encourage staff to help clients implement elements of the reform. It may involve making resources available for applied learning and building a pool of staff with expertise in specific areas, such as coaching, market engagement, use of data for decision-making, and quality and sustainability approaches. In addition, it might be beneficial to recognize procurement staff, task teams, and clients who collaboratively tailor procurement to client needs, apply quality and sustainability approaches, and demonstrate procurement outcomes in the areas of the World Bank’s framework. Fostering collaboration between technical teams and procurement staff to strategize procurement approaches in projects to benefit clients and implement reform elements at scale could also improve support.
• Enhance procurement data systems to benchmark outcomes, identify bottlenecks, and inform decisions to improve reform implementation and project procurement. Adding benchmarks to monitoring dashboards could help track procurement outcomes for the reform principles (that is, efficiency, economy, integrity, fairness, transparency, fit for purpose, and value for money). Monitoring procurement processing issues could help solve these issues more efficiently. Simplifying data entry for clients could facilitate timely and complete information on procurement activities. At the reform level, the data acquired could also be used to correct shortcomings while implementing reform elements. At the portfolio level, the data acquired could be used to prioritize and allocate procurement staff and resources to support procurement achievements in projects with the greatest needs—those with frequent procurement issues, limited experience with planned approaches, low capacity to carry out procurement, and several procurement risks. At the project level, the emphasis could be on providing intensive procurement staff support starting from project preparation through the first year of implementation. For example, the support could help strategize procurement approaches and ensure early contracting to maximize the success toward the project’s development objectives.

Recommendation 2. Strategically strengthen country-level procurement capacity.

Proposed actions are as follows:

• Engage in country-level dialogue to enhance portfolio performance and the uptake of quality, sustainability, and other innovative procurement approaches. This dialogue would help clients and World Bank staff take advantage of procurement synergies across projects and promote capacity strengthening where relevant to enhance the portfolio’s performance. It could help reduce the burden on task teams, procurement staff, and clients to repeat certain types of procurement-related activities, such as aspects of market analyses in each project, and solve procurement bottlenecks for the project portfolio as a whole instead of on a project-by-project basis. For example, it could help solve bottlenecks related to delays at the start of projects due to client procurement specialists not being in place to strategize and process contracts early. Dialogue could also be used to agree with clients on suitable actions to enhance quality and sustainability in the portfolio and identify innovative ways to improve procurement in World Bank–supported projects in the country context.
• Develop country-level capacity strengthening plans to support countries with persistent procurement issues. These plans could have a long-term horizon (based on timely data on project procurement issues and outcomes, such as from projects, complaints, and client feedback). The plans could be tailored to address procurement-related barriers to project and portfolio performance. The plans could emphasize countries with severe or persistent procurement issues and lower procurement capacity. They could focus on actions to tackle persistent procurement issues in the World Bank’s portfolio, with a strong focus on procurement human resources and knowledge sharing, especially in countries where finding local procurement experts is a constant hindrance. Procurement human resource support could build on HEIS and include training programs with other development partners, the development of university curricula on procurement, and collaboration with regional procurement networks. The plans could also involve strengthening national procurement and complaint systems and other actions to streamline and expand procurement approaches that enhance quality and sustainability.

Recommendation 3. Consistently manage the full spectrum of procurement risks to maximize project success.

Proposed actions are as follows:

• Improve procurement risk identification and data to help enhance project implementation and results. Risk identification could become a collaborative discussion with task teams and clients to identify and support practical actions to mitigate risks while encouraging procurement innovations to help enhance project development outcomes. The World Bank could provide tools and workshops for procurement staff to support clients in managing risks, such as strategizing procurement activities, balancing the use of comprehensive prior reviews with simplified post reviews, engaging potential suppliers, developing procurement approaches, and remotely monitoring procurement. Risk data analytics could be modernized to generate a dynamic risk profile on a wide spectrum of risks using procurement information from projects (for example, using data on procurement activities, complaints, historical performance, post review problems, and indicators or flags to track risks). This could ease the timely collation of fragmented data from multiple sources to inform decisions and the heavy burden on staff to enter risk information repeatedly and manually for many projects. Data could track risks related to value, the timing of procurement contracting to support the achievement of project development outcomes, the use of quality approaches for procurement, market engagement, post review oversight, client readiness and experience to carry out planned procurement approaches, and clients’ use of alternative procurement approaches.
• Use risk data to help World Bank procurement management and staff optimize oversight and support to projects. Based on the risk profile, procurement oversight approaches for clients could be customized and simplified. For example, clients with demonstrated procurement capacity and low risks might require less support, and part of their procurement could be done using country systems. In contrast, procurement activities might need frequent oversight for clients with low capacity and high risks. This includes early oversight of post review processes done by clients in projects and more targeted World Bank support for comprehensive prior reviews. Prior review timelines may be shortened by targeting the World Bank’s review to focus on challenging stages of procurement rather than the entire process. The use of third-party review mechanisms could also be expanded for higher-risk procurements, where clients want the assurance of oversight, but prior reviews would be too slow. Managing risks better, with clear guidance to inform decisions, could help procurement staff customize client support and optimize the time they spend supporting clients. Risk profiles may also change over time and vary for different project components and activities.