Organization
World Bank
Report Year
2014
Accepted
Yes
Status
Active
Recommendation

#6. Focus resources on high risk contracts for prior review, reducing emphasis on value thresholds. Improve the use of risk monitoring tools and risk mitigation methods, through the analysis of risk data from PRAMs and post-procurement reviews, for example by clustering to draw out patterns and trends; and through streamlining details of the PRAMs instrument.abc

Recommenation Adoption
IEG Level of Adoption by Year: mar-rating-popup NTNTSH Management Adoption by Year: mar-rating-mng-popup NTNTSC
CComplete
HHigh
SSubstantial
MModerate
NNegligible
NANot Accepted
NRNot Rated
Original Management Response

2015

Action Plans
Action 1
Action 1 Number:
0328-01
Action 1 Title:
Action 6 (a) : Develop strategy/guidance to determine country/sector/agency thresholds based on risk and market profile; define
Action 1 Plan:

Action 6 (a) : Develop strategy/guidance to determine country/sector/agency thresholds based on risk and market profile; define high risk/complex procurement and develop the appropriate level of prior review required

Indicators: Bankwide strategy/guidance developed on risk thresholds

Baselines: Thresholds are low and have limited correlation with risk and principles for setting thresholds vary across Bank regions

Targets: Strategy/guidance developed and revised risk thresholds implemented

Timeline: FY 15.

Action 2
Action 2 Number:
0328-02
Action 2 Title:
Action 6 (b) : Improve the strategic use of post procurement reviews
Action 2 Plan:

Action 6 (b) : Improve the strategic use of post procurement reviews

Indicators: Guidance note prepared on better use of post procurement reviews for risk management.

Baselines: Post procurement reviews are not systematically used.

Targets: Guidance prepared on strategic use of Post procurement reviews

Timeline: FY 15.

Action 3
Action 4
Action 5
Action 6
Action 7
Action 8
IEG Update:
No Updates
Management Update:
No Updates
IEG Update:

Latest progress update on the agreed actions on this recommendation are as under:
(i) This recommendation requires development of strategy/guidance to determine country/sector/agency thresholds based on risk and market profile; and define high risk/complex procurement and develop appropriate level of prior reviewrequired. The New Procurement Framework (NPF) has streamlined prior review to free up resources to support other features of the framework and could save significant procurement resources. Under the NPF, the Bank's procurement prior review is focused on high-risk and/or high-value contracts and is applied through mandatory thresholds. Annex 1 of the Procurement Procedure sets out the new risk-based thresholds for different kinds of contracts that apply both to new projects following the NPF and to existing projects. Specific targets will be set to streamline prior reviews. The approach to streamline prior review is complemented with the strategic use of additional oversight instruments to be applied following a fit-for-purpose approach, for example, integrated fiduciary reviews, HEIS, increased oversight and monitoring on contract management, use of third-party assurance, and risk-based post reviews.
(ii) Improve the strategic use of post procurement reviews: Management has issued a methodology for procurement post reviews, as explained in the Guidance for Procurement Post Review. This methodology, which was informed by IAD's advisory review, is more targeted and risk-focused than the former post review. The guidance outlines the agreed risk-based methodology for sampling contracts to be subject to post review, with an annual target of post-reviewing 10% of contracts in the Bank wide portfolio that have not been subjected to prior review. The strategy also provides opportunity for relying on countries' own supreme audit institutions and other third parties to conduct post review.
In addition, IEG is happy to note that Project Procurement Strategy for Development (PPSD) is intended to be a living documentprepared by the borrower taking into account risks (operational context, client capability, procurement complexity, and market risks) which together with the Bank's analysis of the risk of the project (Procurement Risk Assessment) in turn should inform decisions on Bank's prior reviews and ex-post. It is also noted that in addition to project-level risk assessment, the NPF introduces the concept of activity level risk assessment to drive procurement and fiduciary decisions. Management has reviewed the PRAMS risk assessment methodology and issued updated Guidance to staff. Management has also integrated PRAMS with the STEP system to enhance the ability of staff to make adjustments to the procurement risk ratings and to further ensure transparency and reporting capability of the system.
IEG shall await outcomes of the use of new risk-based prior and post review methodologies.

Management Update:

The NPF has streamlined prior review to free up resources to support other features of the NPF, including country capacity building. Analysis of the framework shows that initiatives to focus prior-review activities can save significant procurement resources (up to the equivalent of 60 Bank staff)resources that can be used, for example, to support HEIS at key stages of the procurement process for operations covered under paragraph 12 of OP 10.00. Specific targets will be set to streamline prior reviews. Under the NPF, the Bank's procurement prior review is focused on high-risk and/or high-value contracts and is applied through mandatory thresholds (Annex 1 of the Procurement Procedure sets out the new risk-based thresholds for different kinds of contracts) that apply both to new projects following the new procurement framework and to existing projects. The approach to streamline prior review is complemented with the strategic use of additional oversight instruments to be applied following a fit-for-purpose approach, for example, integrated fiduciary reviews, HEIS, increased oversight and monitoring on contract management, use of third-party assurance, and risk-based post reviews (discussed below). Management has issued a methodology for procurement post reviews, as explained in the Guidance for Procurement Post Review. This methodology, which was informed by IAD's advisory review, is more targeted and risk-focused than the former post review. The guidance outlines the agreed risk-based methodology for sampling contracts to be subject to post review, with an annual target of post-reviewing 10% of contracts in the Bankwide portfolio that have not been subjected to prior review. The strategy also provides opportunity for relying on countries' own supreme audit institutions and other third parties to conduct post review. Procurement Risk Assessment: the Project Procurement Strategy for Development (PPSD) is intended to be a living documentprepared by the borrower taking into account risks (operational context, client capability, procurement complexity, and market risks) which together with the Bank's analysis of the risk of the project (Procurement Risk Assessment) in turn should inform decisions on Bank's prior reviews and ex-post. In addition to project-level risk assessment (which is currently captured in Project Risk Management System, P-RAM, by risk rating multiple aspects/dimensions relating to the project and/or implementing agency), the new framework introduces the concept of activity level risk assessment to drive procurement and fiduciary decisions. Management has reviewed the PRAMS risk assessment methodology and issued updated Guidance to staff. Management has also integrated PRAMS with the STEP system to enhance the ability of staff to make adjustments to the procurement risk ratings and to further ensure transparency and reporting capability of the system.

IEG Update:

IEG notes the global consultation on prior reviews and concerns about funding issues. IEG has requested and received information on the global consultations including a summary of the second consultation (attached). The summary finds that the proposed new procurement framework would require a culture change among World Bank staff in order to be effectively implemented. World Bank staff would need to understand the intricacies of country contexts, legal systems, and sectoral issues. Consultation participants also pointed out that the World Bankメs procurement function would need to be adequately funded. They agreed that focusing prior reviews on high risk/high value procurements would help free up some of the necessary resources. Stakeholders pointed out that some risk was inherent in an increase in staff discretion, which would need to be mitigated through adequate training and oversight.ヤ

IEG notes that specific conditions for the use of the Borrowersメ bidding documents and processes below ICB are set out in the new procurement framework, Regulations for Borrowers; paragraph 7.20 of the Section VII.

IEG would also like to review the sampling process suggested for procurement that is below the threshold and an update of the agreed final method as agreed with IAD. IEG would also welcome some evidence of instances where countries' national authorities are being invited to participate in post-reviews,

Management Update:

Reflecting IEG feedback, Management has consulted globally on proposals to optimize prior review. Proposals to streamline prior review are currently awaiting Board approval, if agreed a program of prior review optimization will be implemented by the GGP with reporting to OPCS.

The Bank will focus its prior review activity to optimize performance. The new Procurement Framework allows the Bank to focus most of its prior-review resources on high-risk and/or high-value procurements, reducing reviews of lower-value and lower-risk procurements to free up resources.

Management has set an absolute value and risk threshold that determines mandatory prior review, below which prior review should not be carried out (except in fragile and conflict-affected situations and other specified circumstances). For procurements that are not prior-reviewed, Management has outlined a sampling method for post review and will obtain assistance from IAD to refine it. Consultations also showed broad support for the appropriate use of a country's own supreme audit institution and other third parties/civil society organizations to undertake post reviews of procurements.

Management is currently reviewing PRAMS to streamline the approach and to join it up with other risk tools in the proposed new Procurement Framework. Management has already implemented SORT as a mechanism to integrate FM and procurements risks to create a fiduciary profile for a project.

IEG Update:

WB: Partially agree. # 6. The prior reviews are not based on value thresholds only. The approach is a combination of risk and value. However, Management agrees that more focus needs to be placed on higher risk/higher value activities, reducing the review of lower value and/or lower risk contracts. In practice, this will mean reducing the number of prior reviews of lower value and/or lower risk contracts, as such, increasing the Banks risk exposure in these areas. However, Management agrees with IEG that this will have "only limited impact on risk and competition" - in FY13, 80% of prior reviews undertaken (by volume), account for 6% of the total value of these contracts. Management proposes to update its approach to prior-reviews, reflecting IEG's findings, releasing resources so they can focus more effort on the highest risk and/or highest value contracts.

Management agrees with IEG that the institutional system to manage procurement risks can be sharpened in focus, better integrated with other systems, and made better use of in terms of data input and analysis of findings.

PRAMS can be simplified and more guidance to staff on how to use the tool can be provided. Reporting tools can be built to address analytical needs. F&C dimensions of risks can also be more explicit within the PRAMS tool.

Management Update:

The Bank puts considerable emphasis, in terms of its present risk management framework, on ex ante risk controls through mechanisms such as prior review and clearance thresholds, which require contracts above certain values to be prior reviewed, by procurement staff at increasing levels of seniority, depending on contract value. Yet findings show that prior review instruments at best partially reflect country or project risk, and as such, could be relied on less as risk control mechanisms. There is likely to be scope to reduce the risk efficiency trade off by reducing the share of contracts that are prior reviewed and focusing prior review on the highest risk contracts. Bank procurement staff acknowledge that current review procedures are helpful for controlling fiduciary risk. Nevertheless, on balance, there was a view that current prior review threshold could be raised with beneficial impacts on workload and clearance time, and only limited impact on risk or competition.
Additional risk that this may imply could be mitigated by better use of Post Procurement Reviews (PPRs) and Independent Procurement Reviews (IPRs). This need not imply an increase in their numbers but rather, a more strategic use of their findings. There is limited analysis of the effectiveness or outcomes of pivotal risk management tools such as the procurement threshold system, the content of post procurement reviews, or data collected through the most recent tool, the P-RAMS instrument, in terms of analyzing content, tracing trends, or correlating risk management and procurement outcomes.
The P-RAMS template and process can be cumbersome and time consuming, especially with multiple implementing agencies, and exacerbated by multiple sequencing being conducted in far too short a time span. Its template, which limits discretionary response, may contribute to fostering a rigid "check the box" approach which limits added value. Risks identified are not weighted or prioritized and can lead to misleading 'averages' or risk in situations where a small number of high risks dominate overall project risk. While the P-RAMS system has a facility for procurement staff to override automatic ratings, it is not transparent and there is little evidence that it is exercised. The P-RAMS template could also further sharpen its focus on fraud and corruption (F&C) by featuring F&C risk as an additional Risk Factor.