RECOMMENDATION 1: IFC, MIGA and the World Bank should harmonize their SME approaches and make clear the objectives and analytic justification for targeted SME support, how it relates to systemic reform, where it is appropriate, what main forms it will take and how it will be monitored and evaluated.
For countries where SME development is a priority, any targeted support should be firmly grounded in the Country Partnership Framework/strategy, the relevant parts of the Systematic Country Diagnostic based on country analytic work, and other instruments which provide an analytic and strategic framework that identifies the sequence and mix of systemic and targeted interventions that will address systemic challenges to SMEs, building markets and access to services. The specification of the target for TSME projects should relate to country-specific conditions and in some cases address small and medium firms differently based on how they experience existing country conditions. While recognizing the different business models across institutions, shared country strategies that leverage and sequence the expertise and comparative advantages of the WBG institutions should ensure complementarity, maximize impact and reduce the potential for redundancies and inconsistencies. Targeted support for SMEs needs to be firmly rooted in a clear, evidence-based understanding of how the proposed support will sustainably remove the problems that constrain SMEs' ability to contribute to employment, growth and economic opportunity.
The M&E framework should be designed to capture the effect of project interventions in these dimensions - at the beneficiary, client and broader market level. At both levels, information is needed to understand the counterfactual - what would have happened without the project. This means, where possible, a rigorous, fact-based approach that generates information on the baseline, the post-project period, and control group. A longer-term timeframe may be required to collect data to evaluate sustainability of impact.
The evaluation finds that, at present, many targeted projects as defined in the approach paper supporting this evaluation are weakly justified, are weakly focused on SMEs, and/or have limited potential for additionality. Contributing to the resolution of systemic economic constraints - leveling the playing field -- hence to better functioning of markets and institutions would allow SMEs to realize their full potential for generating jobs and growth in developing economies. Systemic priorities also include establishing the legal, regulatory and institutional environment supporting a deep, competitive and stable financial sector, where financial institutions seek SMEs as clients. The scale of gaps identified for SME services, especially finance, dwarfs the direct benefits WBG can deliver, so targeted interventions need to be strategic, leveraging resources to produce broader, sustained benefits for institutions and markets.
Inconsistencies and limited coordination across WBG institutions result in missed opportunities for institutions to leverage each other. The lack of institutional consensus on what constitutes an SME, when it is appropriate to support them, and what constitutes success seems especially inappropriate as the World Bank Group moves towards global practices crossing traditional boundaries under a "One World Bank Group" model.
WBG: Agree. The introduction of Global Practices offers an opportunity to improve the harmonization of the SME approach across the WBG. Thus, the detailed actions that Management will take to achieve this goal will be determined once the new governance structure is in place.
In countries where SME development is a priority, the introduction of SCD/CPF process will also help identify constraints and opportunities at country level that could be addressed by targeted SME support, granting these interventions an evidence-based knowledge of how WBG support could help remove constraints that limit contribution to economic growth and job creation.
IFC's engagement with the Global Practices, as it updates its recent discussion document on SME stocktaking and new directions, can provide a forum for discussion of some of the key issues raised in the report.
With regard to monitoring and evaluation, both IFC and IBRD/IDA are strengthening M&E frameworks for SME and related activities and are also working to harmonize monitoring indicators. Management will explore the potential for further coordination on evaluation of impact from SME support projects, bearing in mind that clients between the two entities can be different in nature.
One important IFC initiative already planned will be the Global SME Finance Facility M&E plan to help test new methodologies and learn about how best to measure outputs and impact. IFC Management will explore M&E efforts focused on intermediate outputs, but also reaching to enhanced evaluative approaches, and impact evaluations. It must be noted that a counter-factual approach will not be feasible in many cases and Management has to be selective on impact evaluations as they are very resource intensive.
MIGA will work with IFC and IBRD/IDA in harmonizing the WBG approach to SMEs and seek clarity in objectives and analytic justification for targeted SME support. With regard to M&E, MIGA will build on and learn from IFC and IBRD/IDA initiatives. MIGA will also examine the M&E framework for SMEs as part of its ongoing internal review of SIP. Together with IEG, MIGA will assess the programmatic evaluation approach used for SIP in the current evaluation and extract Lessons of Experience.
Action 1D (IFC FIG): Implement a M&E program for the Global SME Finance Facility (GSMEF)
Indicator: M&E program for the Global SME Finance Facility available.
Baseline: No M&E âprogram available for the GSMEF.
Target: M&E mid-term and final review for the Global SME Finance Facility completed.
Timeline: Mid-term review 2016, Final review FY19
IEG cannot determine from IFC's response any progress since the prior update, nor was any evidence provided of such progress in terms of the targets. As the original Dalberg report was done at a stage earlier than mid-term,
it would be good to have an actual mid-term review. As the program has been extended, it is not clear that a final review will be conducted in the lifespan of this MAR.
The Indicator is at 80% of the target. The Global SME Finance Initiative was created in March 2012 with the objective of increasing access to finance for small and medium-sized enterprises (SMEs), and thus creating new employment opportunities in some of the world's most challenging economies, such as IDA and FCS. Recently, it was agreed with the donors to extend the term of the Facility until 2021, and additional negotiations are ongoing to further extend it to 2024/26.FIG, together with the Development Impact team, has been continuously working on collecting and analyzing the data from its Facility clients and from SME beneficiaries for the final evaluation. An evaluation of the impact of a client bank's lending to their SME clients in Bangladesh has been completed. Research on the impact of financial services, including lending on another client bank's SME customers, is also underway in Kenya, and more evaluations are in the pipeline. In addition, IFC has improved the 'jobs created' model from last year (by calculating for first time 'jobs supported') and successfully presented it to the donors. Proof of this success is that donors are asking to learn more about the jobs model since they plan to replicate it for their own separate business initiatives (outside the Facility).Please note that we suggest renaming the mid-term evaluation that Dalberg performed to 'formative' since it was done at a stage earlier than mid-term.(Dan Goldblum has completed the recommendation update and rating for the SME MAR as the Task Leader. As the Reviewing Manager, I, Wendy Teleki, have also reviewed and cleared the update and rating for this recommendation and has authorized Abdul-Rahman Akande to clear in the MAR system on my behalf.)
It is difficult to judge the exact state of fulfillment of this action given that there is no new evidence submitted of activity beyond the mid-term review, which was submitted with last year's update. IEG is encouraged that FIG is engaged in ongoing work with the Development Impact Unit and welcomes evidence of progress towards the completion of the action.
Indicator is at 60% of the target. The Global SME Finance Initiative was created in March 2012 with the objective of increasing access to finance for small and medium-sized enterprises (SMEs), and as a result creating new employment opportunities in some of the world's most challenging economies. The IFC contracted Dalberg Global Development Advisors and Beecher Analysis Group (the Evaluation Team) in June of 2014 to evaluate the Initiative's impact with respect to its theory of change and expected results. The mid-term review of the Global SME Facility has been successfully completed by Dalberg. FIG is working with Development Impact Unit to collect data needed for the final evaluation. For example, Dalberg's extrapolation model has been updated by IFC staff and successfully presented to donors additional case studies of facility studies are ongoing.
(Note: Dan Goldblum has completed the recommendation update and rating as the Task Leader. The Reviewing Manager, Wendy Teleki, has also reviewed and cleared the update and rating for this recommendation and has authorized Abdul-Rahman Akande to clear in the MAR system on her behalf)
IEG notes the existence of a mid-term review for the Global SME facility, as evidenced by a summary note, not the mid-term review itself. IEG notes that the summary document received states the the Facility has met or exceeded most goals,. However, the summary document is missing important details, based on which IEG cannot determine whether this reflects a serious and valid M & E program. Without the actual mid-term report, IEG cannot validate the summary statements provided. For example, the summary document says it relies on an "extrapolation" of job creation without describing its basis or method. For example, the summary document does not state average loan size, it simply makes characterizations about trends and tendencies in loan size. For example, the summary document does not state the criterion used for measuring "SME loans" in participating institutions. For example, the document does not state how loans by participating FIs were attributed to the Global SME facility support. For example, the tabular data appears to credit $3.1 billion in SME loans to Facility-sponsored advisory services in FCS countries, but does not explain the rationale for this connection or attribution.
Indicator is at 50%of the target. The Global SME Finance Initiative was created in March 2012 with the objective of increasing access to finance for small and medium-sized enterprises (SMEs), and as a result creating new employment opportunities in some of the world's most challenging economies.
The IFC contracted Dalberg Global Development Advisors and Beecher Analysis Group (the Evaluation Team) in June of 2014 to evaluate the Initiative's impact with respect to its theory of change and expected results.
The mid-term review of the Global SME Facility has been successfully completed by Dalberg.
Next steps: FIG is working with Development Impact to collect the needed data for the final evaluation.