When the report was completed in June 2016, none of the PforR programs had closed, and it was too early to draw definitive conclusions in many areas, but the evaluation has brought to light some important insights from the instrument’s early design and implementation experience.
Growth and Diversification of the PforR Portfolio
From a modest start, the Bank’s PforR portfolio has grown rapidly. As of March 31, 2016, the Board had approved a total of 39 PforR operations, providing US$9.4 billion of Bank financing to support a total of US$49.9 billion in government programs, with an additional 21 operations under preparation (having completed the concept stage), totaling US$5.4 billion in expected Bank financing. As of March 31, 2016, all Bank Regions had at least two approved PforR operations. The introduction of PforR to different Regions appears to be influenced by a range of factors. In Africa, for example, it has been influenced by the perspective that the PforR is a good instrument for supporting the regional agenda of building stronger institutions and delivering better services. In the Middle East and North Africa and East Asia and Pacific Regions, the instrument is regarded as a good fit for efforts to increase the emphasis on results and institutional capacity building. PforR operations cover most of the sectors in which the Bank traditionally provides financing. Water; Social, Urban, and Rural; and Health, Nutrition and Population are the leading sectors.
The PforR Portfolio (Percentage of Operations)
Overall, the structure of the Bank’s assessments for the PforRs has proven to be appropriate. The technical, fiduciary, and environmental and social assessments have generally been credible and comprehensive. The results frameworks, disbursement-linked indicators (DLIs), and program action plans (PAPs) are often reasonably coherent, and risks related to PforR operations have generally been well identified and assessed.
The results frameworks are often reasonably coherent, but their design with the DLIs could be strengthened to present a clearer line of sight to the developmental results. The Program Development Objectives (PDOs) are rarely at the outcome level, and explanations of how the PforR objectives relate to the longer-term objectives of the supported government program are mostly absent from the Program Appraisal Documents(PADs). The DLI indicators are not always well integrated with the results frameworks. In order to ensure a higher likelihood of achieving the ultimately desired developmental results, more consistent linking of the DLIs to the results frameworks and a clearer line of sight to the longer term objectives will be required.
With respect to the design of DLIs, some attention is needed to the balance between the disbursement and incentive/stretch objectives. The instrument’s DLIs are trying to achieve several partially offsetting objectives, including as triggers for disbursements, with a need for predictability, and as incentives for performance towards stretch targets. The report finds initial indications of an inherent tendency to shift the balance towards the disbursement objective, and suggests that continued attention needs to be given to achieving the right balance.
Ownership and partnership have been well addressed in the program documents. The field visits found a considerable degree of government ownership of the programs under implementation. However, there is no evidence yet that the instrument has encouraged a broader use of the strengthened country systems beyond the programs supported by the PforRs, or mobilized much additional financing by other donors. The PAPs have been an important part of the operations, including capacity building, but their implementation has frequently been delayed, and some could have benefited from more clearly defined goals.
It’s too early to derive conclusions about the efficiency of PforRs relative to other Bank Instruments. Both the Bank teams and government counterparts have moved well up the learning curve for this new instrument, and countries have been eager to rely on their own financial management systems and procedures. So far, the Bank’s average costs for the preparation of new programs have been similar to those of other investment project finance (IPF) operations, with significant variations among programs, while average Bank implementation costs have been significantly higher than for IPF operations. There may be increased positive externalities/public good aspects from strengthened country systems. Overall, however, there is not yet sufficient evidence to derive any conclusions about the overall efficiency of PforRs.
The high environmental and social risk exclusion has sometimes been in interpreted in an overly cautious manner. This has led to the avoidance in PforRs of investments that would normally be integral to the supported government programs, which raises a concern of how such investments will be handled if they are not subject to the oversight associated with Bank involvement.
The implementation of PAP actions has often been delayed. Generally, the quality of PAPs at the entry has been satisfactory, although some PAPs could have benefited from stronger focus on key aspects. However, implementation performance has been more uneven. One reason may be that there is in practice no penalty for slow or poor implementation of PAP items, except for the actions linked to DLIs and effectiveness conditions. Based on findings from operations visited by IEG, the implementation of PAP actions has often been delayed, sometimes with important implications for the timely achievement of the programs’ results. But both in the country monitoring processes and in some aide-memoires, there is a tendency to focus on the DLIs, with less systematic attention to the PAP items and also to the indicators in the results frameworks.
The management of risks is progressing well, albeit the reporting system is inadequate. PforRs appear to be making progress with the management of all kinds of program risks. The Implementation Status and Results reports (ISRs), however, do not adequately reflect the frequency of delays, and they provide no information on the environmental and social effects of the operations. This is surprising, since environmental and social risks have been rated moderate or substantial in most PforRs, and the monitoring of impacts is an essential element of the Bank’s environmental and social risk management framework.
Strengthen the design of the results framework and the DLIs to ensure that the PAD presents a clear line of sight to developmental results.
Strengthen the design and monitoring of the Program Action Plans.
Minimize the overly cautious interpretation of the "high" environmental and social risks exclusion.
Strengthen the monitoring and reporting of results to cover systematically the entire results framework, as well as the environmental and social effects of the projects.
Watch the brief presentation: Evaluating the Program-for-Results Lending Instrument