5. Strengthen M&E in Bank and IFC health financing projects by:
- Improving appropriate M&E frameworks in Bank and IFC projects to put in place mechanisms to collect and monitor relevant indicators and
- Monitoring distributional indicators, including on access and outcomes, consistent with benchmarking and tracking progress towards Universal Health Care coverage.
abc
Evidence is scant on the effect of Bank and IFC operations and programs on final outcomes.
The quality of project M&E is weak in Bank and IFC health projects. However, there is an increase in the number of impact evaluations.
The HNP strategy stipulates that the Bank monitor how health financing affects equity in service use, risk pooling, and financial protection, but this information is rarely collected in health financing operations.
WB, IFC: Agree. A Universal Health Care monitoring framework has been co-produced with WHO to monitor access to essential services, the level of financial protection granted to the population and equity in health care. WB: Agree. Given IFC's limited exposure in the health financing space, IFC Management generally agrees with the IEG's recommendation to strengthen M&E frameworks and will track indicators that measure effectiveness when investing in this type of projects or conducting impact evaluations in the future.
The GP will continue to improve the monitoring of health financing projects.
Action 8: Improve monitoring of HF projects, including distributional outcomes.
Indicator: % of active operations/investments with HF activities that adopt performance indicators consistent with WBG/WHO UHC monitoring framework , including baselines and targets.
Baseline: TBD (FY2015)
Target: TBD
Action 8: Improve monitoring of health financing projects, including distributional outcomes.
The indicator selected to monitor progress under Action 8 is "the percentage of active operations/investments with health financing activities that adopt performance indicators consistent with WBG/WHO UHC monitoring framework". In FY15 (the baseline year) 86% of active operations/investments with health financing activities had adopted performance indicators consistent with WBG/WHO UHC monitoring framework. The indicator had already reached the 90% in FY16 (reaching the target that was expected to be reached by FY18). The 90 percent rate was maintained in FY17.
IEG agrees that the Monitoring and Evaluation (M&E) frameworks of recently approved health financing projects are usually consistent (with few exceptions) with the WBG/WHO UHC monitoring framework, even if the frameworks are not always able to monitor the distributional aspects of the expected results and outcomes.
Finally, the importance of the partnership with WHO in developing a unified UHC monitoring framework, the work conducted to develop a global UHC monitoring database of standardized indicators for service coverage and financial protection, and the training provided on UHC measurement using the ADePT software are acknowledged as factors that contributed to improve the monitoring of health financing projects.
[Action 8: Improve monitoring of HF projects, including distributional outcomes. Indicator: % of active operations/investments with HF activities that adopt performance indicators consistent with WBG/WHO UHC monitoring framework. FY15 (baseline): 86% FY16: 90% FY17: 90% FY18 (target): 90%].
The WBG's collaboration with the WHO, as well as the HF GSG's staff capacity-building activities, support the successful monitoring of health financing projects, including for distributional outcomes. Results for Action 8 show that 90% of active operations with health financing activities in FY17 use performance indicators consistent with the WBG/WHO UHC monitoring framework.
Through the WBG/WHO partnership, the UHC monitoring framework was developed to help countries track progress and results towards UHC. The HNP GP has continued to develop the UHC monitoring database of standardized indicators for service coverage and financial protection, conducting analytical work to populate these indicators, and piloting training on UHC measurement using the ADePT software. The UHC monitoring database was developed with data from 873 household surveys and continues to cover 88% of the world population for service coverage indicators, and 89% of the world population for financial protection indicators. Annual updates are published in the WBG/WHO Annual UHC Monitoring Report. This report provides analysis on the progress toward UHC on the country level globally and includes a breakdown by distributional outcome. The output of the report ensures that the impact of HF projects in client countries is closely tracked and that an analysis by income level is consistently available.
In addition to its work with WHO, the HNP GP collaborates with the IFC to ensure that any new health financing work is consistent with WBG goals for monitoring and outputs. Further avenues of work in this area will include support for increased focus on health financing-related activities in IFC, supported by the IDA18 Private Sector Window.
In addition, similarly to the internal capacity development and dissemination of good practices for analytical work seen in Action 3 (poverty/shared prosperity focus), the HF GSG's clinics and peer review activities also help improve monitoring for distributional outcomes in health financing projects. The good practices gathered through the HF GSG are shared with the WBG staff through capacity-building activities, which in turn reflect the emphasis on understanding the impact of health financing work on the bottom quintiles of countries. These activities result in an improvement in the monitoring of the effect of health financing projects across the income distribution metrics.
The WBG improved the monitoring of the distributional outcomes of health financing projects. Including the portfolio review of M&E arraignment of active health financing operations and of the Health in Africa Initiative. IEG agrees that the global UHC database with standardized health financing indicators covering 88% of the world population for service coverage indicators, and of 89% of the world population for financial protection indicators will enhance the M&E framework of health financing projects.
The WBG and WHO jointly developed a monitoring framework to help countries track progress and results in UHC, and the HNP GP has sustained its focus on improving monitoring of health financing projects, including for distributional outcomes, in both FY15 and FY16. Accordingly, the portfolio review shows that in FY16 90% of active operations with health financing activities use performance indicators consistent with the WBG/WHO UHC monitoring framework. Similarly, the majority of the Health in Africa Initiative Advisory Services projects includes indicators consistent with the WBG/WHO UHC monitoring framework. The HNP GP in FY16 further conducted a set of activities to support its commitment to monitoring UHC, including building a global UHC database of standardized indicators for service coverage and for financial protection, piloting training on UHC measurement using the ADePT software, and conducting analytical work to construct global estimates for financial protection indicators. The construction of a global UHC database with standardized indicators was achieved using data from 873 household surveys. Such database covers 88% of the world population for service coverage indicators, and of 89% of the world population for financial protection indicators. As IFC's work on health financing further develops over the coming couple of years, close coordination is planned to ensure continued strengthening of the WBG M&E capacity and outputs in this area.
[Action 8: Improve monitoring of HF projects, including distributional outcomes. Indicator: % of active operations/investments with HF activities that adopt performance indicators consistent with WBG/WHO UHC monitoring framework. FY15 (baseline): 86% FY16: 90% FY18 (target): 90%].
The Global Practice (GP) is preparing to increase its focus on strengthening its M&E and monitoring the distributional impacts of its operations. To date, no progress has been reported and the Global Practice has not set the baseline or targets for this indicator.
The HNP GP is planning/preparing several activities to strengthen the monitoring of projects, including HF-related operations and distributional outcomes:
1. A plan to strengthen M&E of HNP operations has been developed, including additional expert support available to country teams.
2. Work is underway toward an enhanced peer review process to strengthen operations, including their M&E frameworks.
Discussions between WB and IFC to coordinate the strengthening of M&E frameworks are planned for November.
The review of operations/investments active in FY15 is ongoing, hence, baselines and targets for the performance indicator are missing.