Recommendation 6: The World Bank Group should define principles for the monitoring of PPPs over the long run, i.e., beyond operational maturity (IFC/MIGA) and projects closure (WB), to capture all vital performance aspects of PPPs, including - where relevant - user aspects.
This evaluation found that evaluation reports that shed light on important aspects of public service delivery are rare, for instance, on access, pro poor aspects, and quality of service delivery.
The existing monitoring and evaluation systems
(Expanded Project Supervision Reports,
Implementation Completion and Results Reports,
Project Evaluation Reports, and so forth.), IFC's Development Goals and DOTS do not record these data systemically. In light of the Bank Group's central goal of fighting poverty - reaffirmed by the new 2013 strategy's twin goals
of ending extreme poverty and promoting shared prosperity, and in light of the intent to
increasingly pursue PPPs, there is an urgent need to introduce a more systematic way of monitoring PPPs. Such a system should not only better capture the end-user aspects of PPPs (when relevant), but should also monitor PPP performance beyond the early years of operational maturity.
As monitoring and evaluation systems are
resource intensive and need to be embedded in corporate reporting systems - which should in any case collect the relevant outcome data on a regular basis - and national statistics services.
WB: Agrees. The WBG is focused on decreasing the incidence of absolute poverty and boosting shared prosperity. Access to basic services remains far from universal across the developing world, which lowers quality of life and can constrain productive activities. Accordingly, the WBG has identified PPPs as an important delivery mechanism to maximize the reach of public resources, while improving the efficiency and quality of the basic services reaching citizens.
Management agrees with IEG that monitoring the effects of WBG PPP operations is vital. IFC MIGA and the World Bank will work together to identify a process through which a suite of principles can be created to guide and inform task teams seeking to monitor the performance of PPP operations. Additional work on impact evaluation placing PPPs against other models of service delivery may have to be explored for a fuller understanding of potential impacts.
WB Action 6: More consistent approach to the assessment of impacts of PPPs across the WBG.â
Indicator: (a) More consistent approach to long term monitoring of PPPs across the WBG.
(b) Summary assessments of impact of PPPs, including on improved services and poverty reduction, and PPP interventions by WBG based on a sample of projects.
Baseline: IFC and PPIAF currently undertaking work looking at the impact of their activities post completion
Target: (a) Develop common WBG assessment methodology for assessments of impacts of PPP projects
(b) Summary assessment of the impacts of a sample of WBG PPPs
Timeline: (a) End October 2015
(b) End June 2016.
The Operational Performance Monitoring (OPM) system for PPPs has been developed based on a comprehensive review of practices across WBG. This study pointed at gaps that need to be closed and suggsts an implementation plan. The develoepd OPM has been tested on projects completed in FY18. Hence, the implementation status can be rated as high.
Pending action: Incorporation of the OPM in he M&E framework of the Cascade approach.
A study to develop more systematic Operational Performance Monitoring (OPM) of PPPs has been completed. This study has reviewed current practice against IEG recommendations and proposed a more systematic, yet practical approach to reporting the actual versus expected outcomes of discrete PPP projects across the WBG. This methodology has been tested on projects completed in FY18 to assess practicality of the approach therefore impacts collected for key areas identified in the IEG evaluation. Application of the approach will be incorporated into the monitoring framework for the cascade.
While the Management update reports on progress, the M&E system is yet in a draft stage and piloting has not yet started. Once piloting has been conducted, the M&E system has still to be formally adopted and subsequently mainstreamed. Likewise, the summary assessment of the impacts of PPP project has not yet started, according to the Management update. Hence rated moderate.
A study to develop more systematic Operational Performance Monitoring (OPM) of PPPs has been commissioned. This study has reviewed current practice against IEG recommendations and proposed a more systematic, yet practical approach to reporting the actual versus expected outcomes of discrete PPP projects across the WBG. Draft approach was developed by June 2016 and the approach will be tested by November 2016.
According to the Management Update, implementation of recommendation 6 has started and some ground work has been done (review of indicators currently used etc.). It is promising to see that there is a plan going forward, hence the implementation status was not rated "negligible", but "moderate" even though the implementation is delayed by some 6-12 months.
During FY15 a review was undertaken of the indicators used in WB PPP lending projects, which showed wide dispersion. During FY15 OPCS initiated a wider review of all indicators used in WB projects. The indicator review will be updated in H1 FY16, drawing upon the quality-reviewed updated tagging of PPP projects in WB/IFC/MIGA. A literature review was also undertaken of links between infrastructure/PPPs and economic growth, income distribution and poverty alleviation, which will help inform the indicators and assessments. Overall, the work in this area is delayed by 6-12 months.