RECOMMENDATION 4: Relevant WBG Management should provide guidance and quality control so that every project documents for WBG projects targeting SMEs will:
a) Define the group of firms to benefit by measurable criteria such as number of employees and annual revenues
b) Justify the definition of the beneficiary group targeted (which could be a subset of SMEs) based on country-specific evidence that this group suffers from size-specific market failures or constraints.
c) Specify and wherever appropriate embed in legal provisions the mechanism to reach the targeted group
d) Include in its results framework and M&E framework indicators of the impact of the project on the targeted group and on the constraints or market failures justifying the project.
e) Projects that describe themselves as targeting benefits to SMEs should reflect this approach. In addition, these projects should be coded accurately with regard to whether or not their benefits are in fact predominantly or exclusively available to SMEs. Coding systems and practices should be reviewed and modified to assure that targeted SME projects are correctly coded, to reduce "false positives" and "false negatives".
Selectivity is required for both efficacy of targeting and its efficiency. The definition of SMEs (both the "ceiling" and the "floor") establishes projects' relevance to development objectives and differentiates some firms from others based on criteria of employment, sales and assets. IFC and MIGA's global definitions appear ill-tuned to many local contexts, while the Bank's lack of any institutional definition can lead to project-specific definitions not firmly linked to the underlying rationale for the assistance offered. In addition, currently, only a minority of projects defines SME (who is eligible for benefits) and fewer still apply that definition through their provisions. A minority identifies the market or policy failure they are seeking to address and a smaller minority provides a solid rationale for how the project will ameliorate that failure. Limited relevant information on portfolio performance makes it difficult to learn from experience or even to establish the existence of additionality of WBG interventions.
IEG's review of project coding suggests a high degree of imprecision in coding projects, including a higher percentage of "false positives" in IFC and "false negatives" in the World Bank.
WBG: Agree. Management agrees with the overall thrust of IEG's recommendations. Most of the detailed recommendations repeat previous recommendations. Thus Management comments provided on the related recommendations are also relevant in this case.
The newly established Global Practices will be accountable for enhancing guidance and quality control for SMEs work.
Coding for SME support will be reviewed as necessary. The forthcoming Guidance Note for Financial Intermediary Financing will also be useful in improving the design and consistency of TSME activities where they involve such financing arrangements.
IFC will continue to strengthen the quality of board and legal documents and will provide guidance to staff on (i) confirming in board papers where the standard definition of SME is being used or an alternative is being used (ii) strengthening the justification for targeting SMEs in Board papers and (iii) including, where appropriate, definitions and provisions in legal documents pertaining to the application of funds for SME on-lending. Regarding the justification for targeting SMEs, it should be noted that IFC's ability to provide evidence on market failures and financing gaps is dependent on the quality of data and analysis undertaken elsewhere, something which will only improve over time.
IFC will continue to improve its mechanism for flagging MSME projects to improve data quality and relevance, recognizing however that the MSME flag for IFC differs from the IEG definition in purpose and composition.
MIGA will strengthen the quality of Board and legal documents and will provide guidance to staff on: the standard definition of SME is being used or an alternative is being used strengthening the justification for targeting SMEs in Board papers and, where appropriate, including definitions and provisions in legal documents pertaining to the application of funds to for SME on-lending.
Action 4A: [WB] Enhancing quality control for SME interventions.
Indicator: Operations with SME support will provide a strong rationale for the intervention, will clearly define the group of firms to benefits, and include clear monitoring indicators.
Baseline: not available
Target: At least 75% of PADs for SME operations will include rationale for intervention rationale and clear monitoring indicators.
Timeline: FY19
Action 4B [WB]: F&M to improve design and consistency of TSME activities.
Indicator: Development of Guidance Note for Financial Intermediary Financing (under OP10).
Baseline: No guidance note exists.
Target: Guidance note developed, and actively disseminated and applied.
Timeline: FY 15
IEG notes the portfolio review conducted as well as the guidance note for financial intermediary financing. Given that the target for 75% of PADs including a rationale and clear monitoring indicators covers the period until fy19, IEG believes more than one year of monitoring evidence would be appropriate for completion of this action, so action 4A should remain active. Action 4B is fulfilled by the issuance and dissemination of the Guidance Note for Financial Intermediary Financing, and could be made inactive.
We reviewed the portfolio of 77 projects active or under preparation in FY16-17 and 77% of operations provide a rationale for the intervention, and define the specific group/subset of SMEs to benefits, and include monitoring indicators. The attached excel file entitled "SME Project database" provides a comparative table with relevant information on every project reviewed.The Guidance Note for Financial Intermediary Financing (under OP10) was updated in March 2016 (revisions attached).
IEG notes that action 4B has been completed through the issuance of the guidance note and dissemination, including through a short note and other means.
IEG notes that there is no evidence supplied suggesting progress towards the target on action 4E (75% of PADs will include rationale for intervention and clear monitoring indicators), and that the framework the action would be based on has not been approved yet.
Action 4A. The development of monitoring indicators to ensure quality control of SME interventions is in process. A draft M&E Framework has been developed for the F&MGP and this is currently under discussion within the Leadership Team of the Global Practice. It is envisaged that this will be rolled out in FY17. In addition, in FMGP a discussion has been initiated around using SME financing as a tool of a counter cyclical policy -- to support employment retention and businesses during a down turn. This will be further elaborated in FY17.
Action 4B. F&M delivered a revised OP10 Guidance Notes in March 2016 a Guidance Note ( attached) for Financial Intermediary Financing -- prepared in conjunction with OPCS. This has been widely distributed within the GP and beyond.
IEG notes the absence of evidence provided on actions 0342-01 and 0342-02 in spite of IEG's request. Nonetheless, the web publication of the Quick Learning note entitled "OP10 and Lines of Credit: Raising the Bar without Raising the Barriers" reflects a serious effort to proviie staff guidance to staff on OP10 and to disseminate the OP10 guidelines., evidencing progress on action 4B.
Finance and Markets GP and Trade and Competitiveness GP have intensified their quality support activities of World Bank projects. Having a common M&E framework with a list of standard indicators to be applied to targeted SME lending and advisory operations across T&C and F&M will greatly support the ability to evaluate progress in the future.
Lines of credit are being reviewed by a group of seven qualified peer reviewers for their compliance with OP10 guidelines. The Global Lead is also attending most SME line of credit operations as a peer reviewer. However, little progress has been made in defining the group of firms -- and justifying the definition of the beneficiary group. Many World Bank projects in other Global Practices (agriculture, urban, energy) continue to operate with lines of credit which may, or may not, be flagged as such for OP10 review. This issue of flagging is currently under discussion with OPCS. As per ADM conventions, it is important to reiterate that the role of the OP10 reviewers is merely as "advisors" -- without compunction by the TTLs to follow the advice provided. We are currently reviewing the OP10 Guidance Note -- but the initial sense is that it continues to convey the type of guidance that needs to be conveyed for line of credit operations.
Additional Information: See attachments