Ensure attention at project inception to evaluability through (1) developing standards for baseline measurement, (2) explicit linking of the baseline to indicators relevant to project objectives, including any that refer to poverty or inclusion impacts, and (3) robust planning for monitoring data required for expost evaluation.
Deficiencies in the M&E design for projects are most frequently identified as shortcomings in Bank support at entry. Often what gets measured are the project milestones (mainly outputs or intermediate outcomes) but often not the final outcomes. Early attention to project evaluability should strengthen M&E planning and execution.
WB: Partially Agrees. Management agrees with the overall thrust of the recommendation to strengthen the project's M&E framework, but finds the specific actions recommended hard or impractical to translate into monitorable actions.
Establishment of key monitoring indicators that reflect intended project outcomes, with baseline values, at the outset of the project is critical to assessing project impact and monitoring progress. To the extent feasible, such data monitoring should be collected throughout the project implementation in a routinized and transparent fashion. Project should also have to state, at the decision review stage, how it plans to track key indicators and outcomes throughout the project's life.
Strengthening the project's M&E framework and its implementation is a major agenda which will continue to be addressed through enhanced guidance and training for staff, and signaling from corporate and front line management.
Action 9a: Revision and harmonization of the list of the Corporate Results Indicators (CRI).
Indicator: Number of CRI
Baseline: 200+ Core Sector Indicators (CSI) currently tracked
Target: The revised and harmonized list of CRI will comprise of no more than 30 indicators. The indicators will clarify purpose, governance, accountability, and set out a quality control process, being directly relevant to corporate reporting and external commitments in the WBG/WB Corporate Scorecards and IDA Results Measurement System.
Action 9b: Develop guidance and good practice notes for Task Teams on development and use of indicators to track project outcomes.
Indicator: Guidance and good practice notes in place on the development and use of Results Framework Indicators.
Baseline: Lack of clear guidance for developing project indicators and clarity regarding use of indicators to articulate the results of project interventions.
Target: Guidance and good practice notes issued, complemented by a list of indicators, on which task teams can draw for results frameworks, with standardized definitions and methodologies for each practice/sector.
Action 9a: The action has been completed, as the twenty-five Corporate Results Indicators (CRIs) have been endorsed and launched in May 2017. Action 9.b: The guidance to the Corporate Results Indicators was issued in November 2016. The rating of Substantial, instead of High, is because the good practice notes have not been produced yet (although steps have been taken to produce a first one) and neither has the standard list of project indicators related to project outcomes.
Action 9a: The twenty-five Corporate Results Indicators (CRIs) identified for the purpose of reporting on corporate and external commitments were endorsed and launched by management in May 2017. They have been formally disseminated to staff and the roll-out has proceeded smoothly without any major bottlenecks. Feedback from TTLs and operations staff has been positive. Through the Operations Help Desk, OPCS continues to support operations staff to address specific concerns on the use of CRIs. A summary of the first four months of implementation (June to September 2017) will be shared with staff and management by the end of September 2017.
Action 9.b: The process is underway for developing the results measurement good practice notes. The first note will be focusing on monitoring results in Country Programme Frameworks (CPFs). OPCS commissioned a consultant to conduct an assessment of the current practices to better understand gaps, needs and good practice examples. The consultant's report was finalized in June (see attachment). The next step involves brief consultations and development of the good practice note. This will be followed by other good practice notes for results measurement at the project level. The development of a standard list of project indicators related to project outcomes is pending but will start in quarter 2 of FY18.
Action 9a: As stated by management, a set of 25 CRIs have been defined and are in the process of being adopted. IEG notes however that there is no indicator that refers to poverty or inclusion impacts (as highlighted in the original recommendation) or a discussion of how the proposed CRIs may contribute to measure poverty and inclusion impacts. The Procedures are very clear on who is supposed to propose, review, advise, clear and communicate the indicators and their future revisions, but are not as specific in assigning specific responsibilities at the level of implementation. The quality control process is not defined at this stage, except for the Procedures indicating that "PGs and Global Practices (GPs) [have] direct responsibility for quality assurance procedures" and that OPCS is coordinating the overall quality control.
Action 9b is at a very early planning stage. Management has shared the TORs referring to an activity, which is about to start, aimed to generate information and evidence on good practices that can be integrated into the CPF module of the RMES Applied Results Measurement Training and used in preparing a good practice note on monitoring CPFs.
The negligible rating is due to the fact that the CRI indicators haven't yet been rolled out and Action 9b has yet to start.
Action 9a: Twenty-five Corporate Results Indicators (CRIs) have been identified for the purpose of corporate reporting and external commitments. Guidance and Procedure notes have been prepared for the CRIs, documenting the purpose, governance, accountability, and set out a quality control process. The CRIs are undergoing the final clearance by management, and are scheduled to be formally disseminated to staff in September 2016, with implementation beginning in January 2017.
Action 9b: The action is still in the early design and planning stage. The team to work on this activity will be assigned and the deliverables, work breakdown structure and task implementation schedule will be discussed and agreed by the end of October 2016.