Organization
World Bank
Report Year
2014
1st MAR Year
2015
Accepted
Yes
Status
Active
Recommendation

Focus resources on high risk contracts for prior review, reducing emphasis on value thresholds. Improve the use of risk monitoring tools and risk mitigation methods, through the analysis of risk data from PRAMs and post-procurement reviews, for example by clustering to draw out patterns and trends; and through streamlining details of the PRAMs instrument.

Recommenation Adoption
IEG Level of Adoption by Year: mar-rating-popup SNTNTNT Management Adoption by Year: mar-rating-mng-popup SNTNTNT
CComplete
HHigh
SSubstantial
MModerate
NNegligible
NANot Accepted
NRNot Rated
Findings Conclusions

The Bank puts considerable emphasis, in terms of its present risk management framework, on ex ante risk controls through mechanisms such as prior review and clearance thresholds, which require contracts above certain values to be prior reviewed, by procurement staff at increasing levels of seniority, depending on contract value. Yet findings show that prior review instruments at best partially reflect country or project risk, and as such, could be relied on less as risk control mechanisms. There is likely to be scope to reduce the risk efficiency trade off by reducing the share of contracts that are prior reviewed and focusing prior review on the highest risk contracts. Bank procurement staff acknowledge that current review procedures are helpful for controlling fiduciary risk. Nevertheless, on balance, there was a view that current prior review threshold could be raised with beneficial impacts on workload and clearance time, and only limited impact on risk or competition.

Additional risk that this may imply could be mitigated by better use of Post Procurement Reviews (PPRs) and Independent Procurement Reviews (IPRs). This need not imply an increase in their numbers but rather, a more strategic use of their findings. There is limited analysis of the effectiveness or outcomes of pivotal risk management tools such as the procurement threshold system, the content of post procurement reviews, or data collected through the most recent tool, the P-RAMS instrument, in terms of analyzing content, tracing trends, or correlating risk management and procurement outcomes.

The P-RAMS template and process can be cumbersome and time consuming, especially with multiple implementing agencies, and exacerbated by multiple sequencing being conducted in far too short a time span. Its template, which limits discretionary response, may contribute to fostering a rigid "check the box" approach which limits added value. Risks identified are not weighted or prioritized and can lead to misleading 'averages' or risk in situations where a small number of high risks dominate overall project risk. While the P-RAMS system has a facility for procurement staff to override automatic ratings, it is not transparent and there is little evidence that it is exercised. The P-RAMS template could also further sharpen its focus on fraud and corruption (F&C) by featuring F&C risk as an additional Risk Factor.

Original Management Response

WB: Partially agree. # 6. The prior reviews are not based on value thresholds only. The approach is a combination of risk and value. However, Management agrees that more focus needs to be placed on higher risk/higher value activities, reducing the review of lower value and/or lower risk contracts. In practice, this will mean reducing the number of prior reviews of lower value and/or lower risk contracts, as such, increasing the Banks risk exposure in these areas. However, Management agrees with IEG that this will have only limited impact on risk and competition - in FY13, 80% of prior reviews undertaken (by volume), account for 6% of the total value of these contracts. Management proposes to update its approach to prior-reviews, reflecting IEGs findings, releasing resources so they can focus more effort on the highest risk and/or highest value contracts.
Management agrees with IEG that the institutional system to manage procurement risks can be sharpened in focus, better integrated with other systems, and made better use of in terms of data input and analysis of findings.
PRAMS can be simplified and more guidance to staff on how to use the tool can be provided. Reporting tools can be built to address analytical needs. FC dimensions of risks can also be more explicit within the PRAMS tool.

Action Plans
Action 1
Action 1 Number:
0328-01
Action 1 Title:
Action 6 (a) : Develop strategy/guidance to determine country/sector/agency thresholds based on risk and market profile
Action 1 Plan:

Action 6 (a) : Develop strategy/guidance to determine country/sector/agency thresholds based on risk and market profile; define high risk/complex procurement and develop the appropriate level of prior review required

Indicators: Bankwide strategy/guidance developed on risk thresholds

Baselines: Thresholds are low and have limited correlation with risk and principles for setting thresholds vary across Bank regions

Targets: Strategy/guidance developed and revised risk thresholds implemented

Timeline: FY 15.

Action 2
Action 2 Number:
0328-02
Action 2 Title:
Action 6 (b) : Improve the strategic use of post procurement reviews
Action 2 Plan:

Action 6 (b) : Improve the strategic use of post procurement reviews

Indicators: Guidance note prepared on better use of post procurement reviews for risk management.

Baselines: Post procurement reviews are not systematically used.

Targets: Guidance prepared on strategic use of Post procurement reviews

Timeline: FY 15.

Action 3
Action 4
Action 5
Action 6
Action 7
Action 8
2018
IEG Update:
No Updates
Management Update:
No Updates
2017
IEG Update:
No Updates
Management Update:
No Updates
2016
IEG Update:
No Updates
Management Update:
No Updates
2015
IEG Update:

IEG notes the global consultation on prior reviews and concerns about funding issues. IEG has requested and received information on the global consultations including a summary of the second consultation (attached). The summary finds that the proposed new procurement framework would require a culture change among World Bank staff in order to be effectively implemented. World Bank staff would need to understand the intricacies of country contexts, legal systems, and sectoral issues. Consultation participants also pointed out that the World Bank's procurement function would need to be adequately funded. They agreed that focusing prior reviews on high risk/high value procurements would help free up some of the necessary resources. Stakeholders pointed out that some risk was inherent in an increase in staff discretion, which would need to be mitigated through adequate training and oversight.

IEG notes that specific conditions for the use of the Borrowers bidding documents and processes below ICB are set out in the new procurement framework, Regulations for Borrowers; paragraph 7.20 of the Section VII.

IEG would also like to review the sampling process suggested for procurement that is below the threshold and an update of the agreed final method as agreed with IAD. IEG would also welcome some evidence of instances where countries' national authorities are being invited to participate in post-reviews,

Management Update:

Reflecting IEG feedback, Management has consulted globally on proposals to optimize prior review. Proposals to streamline prior review are currently awaiting Board approval, if agreed a program of prior review optimization will be implemented by the GGP with reporting to OPCS.

The Bank will focus its prior review activity to optimize performance. The new Procurement Framework allows the Bank to focus most of its prior-review resources on high-risk and/or high-value procurements, reducing reviews of lower-value and lower-risk procurements to free up resources.

Management has set an absolute value and risk threshold that determines mandatory prior review, below which prior review should not be carried out (except in fragile and conflict-affected situations and other specified circumstances). For procurements that are not prior-reviewed, Management has outlined a sampling method for post review and will obtain assistance from IAD to refine it. Consultations also showed broad support for the appropriate use of a countrys own supreme audit institution and other third parties/civil society organizations to undertake post reviews of procurements.

Management is currently reviewing PRAMS to streamline the approach and to join it up with other risk tools in the proposed new Procurement Framework. Management has already implemented SORT as a mechanism to integrate FM and procurements risks to create a fiduciary profile for a project.