Reform the ICR system and its validation to make it more compatible with innovation and course corrections as the report explains. Project teams should be able to change course faster and more often. The ICR system should better account for unintended positive and negative outcomes, beneficiaries' perspectives, and unforeseeable shocks in how results are measured and projects are rated (applies to the World Bank and to IEG's role in validation).abc
The planned reform of the ICR process, template, and guidelines is an opportunity to correct the incentives and signals surrounding self-evaluation, building on the heightened attention that management has started to pay to results frameworks. Staff perceive that the prevailing interpretation of the IEG/OPCS harmonized objectives-based approach to rating and validating ICRs limits the appetite for innovation and causes inflexibility for project management. Adaptability can be promoted through increased flexibility in project design that minimizes the need to amend legal agreements as well as through simplified Bank and client restructuring procedures. There is a need to promote more constructive interactions between IEG and operational departments over project validations without losing sight of IEG's accountability function. Something that would help with this would be a mechanism to flag up when unsuccessful outcomes are caused by major shocks outside the control of the Bank such as, for example, disasters, conflict, and economic crises. The harmonized ICR rating and validation guidelines give insufficient attention to beneficiaries' views and to unintended positive and negative consequences.
World Bank Management agrees on the need to reform the ICR to improve this important self-evaluation tool and enhance its role in learning. Management has already initiated the process of reforming the ICR, aiming to make the evaluation approach clearer and more straightforward, with a simpler format and stronger results focus. It will provide strengthened guidance on better capturing relevant lessons learned and more effectively evaluating pilots and innovative projects. OPCS is planning to hold extensive consultations with the Bank's operations staff and IEG to address the weaknesses identified in the report and to better balance the ICR's learning and accountability aspects.
WB Action 1.1: Develop a revised ICR format, in collaboration with IEG, that makes it more intuitive for Task Teams to describe what was learned through the project and to better highlight lessons that can be applied to future projects [FY17 Q4]
WB Action 1.2: Make changes to ICR guidelines & validation process, in collaboration with IEG, to recognize unintended positive and negative outcomes risk taking and the effects of external shocks beyond the control of project teams [FY17 Q4]
WB Action 1.3: Clarify the ICR âsplit ratingâ system for restructured projects to encourage more timely and more frequent course corrections, if warranted [FY17 Q4]
WB Action 1.4: Explore simplification of project restructuring processes to reduce the level of effort required to carry out course corrections [FY18]
WB Action 1.5: Explore new instruments that would allow more adaptable approaches (e.g. multi-phased programs) [FY18]
WB Action 1.6: Explore ways to enhance course corrections and close feedback loops, including through better articulation of results chains (in the PAD) and more rigorous follow-up through enhanced Mid-Term Reviews [FY18]
We appreciate that management has completed the ICR reform, simplified the approval of level 1 restructurings, and introduced the MPA. Work remains to improve the use and the usability of the ICR as a tool and a mechanism for learning. As explained in the evaluation, this will require much more than changes to tools and processes.
1.a. Key changes introduced as part of the ICR reform includes (i) streamlined processing (ii) reduced number of ratings (iii) simplified and more intuitive structure for capturing lessons learned (iv) consolidated criteria for the outcome ratings and )v) focused and clearer guidelines.1.b. WB Action 1.4. The Multiphase Programmatic Approach (MAP) was approved by the Board in July 2017, which allows clients to structure a long or complex project into a series of phases. Basically, this new approach makes better use of our financial resources, encourages more learning and adaptive thinking, and ensures our operations are more responsive to changing country circumstances. A series of operational clinics and informational sessions are being held on MPA. New systems, procedures, and guidance will be made available in FY18.WB Action 1.5. Project Restructuring Policies and Procedures were streamlined to speed up the response time to clients who request level-one restructurings. The Board has agreed in July 2017 to delegate to Management the authority to approve: (i) restructurings that involve Project Development Objective changes and (ii) IPF restructurings that involve triggering of a new safeguard policy. Such restructurings will now be approved by Regional Vice Presidents. These changes will significantly reduce the number of level one restructurings that require Board approval.WB Action 1.6. As part of the effort to simplify the Project Appraisal Document (PAD), OPCS has experimented with the use of the shorter format that is more results focused and user-friendly, including a section on the articulation of the results chain. In the next coming months, OPCS will work with ITS to integrate the new ICR format into the portal. New guidance and series of operational clinics and informational sessions will be held on new PAD in FY18.