Review the costs and benefits of fully applying the DOTS and XPSR frameworks to the GTFP instrument. An annual program level evaluation that includes relevant indicators of additionality and effectiveness could be considered.
4. There are a range of challenges with adapting the evaluation frameworks used for long-term loans and equity investments. In particular: (i) applying the DOTS framework places a heavy data gathering and reporting burden on client banks that may not be commensurate with the relatively limited nature of the instrument and (ii) there is no clear logical link between many of the results indicators in the framework and the trade finance guarantee or advisory services instruments. Attributing results to the GTFP instruments is therefore difficult.
Original Response: Agreed: IFC highly values the approach designed for GTFP's DOTS program and has made significant progress in its implementation (less than two years). IFC is the first and only institution in the world to measure development results of trade finance. IFC agrees to review the costs and benefits of fully applying the DOTS and XPSR frameworks to the GTFP instrument, and will consider the approach of an annual program level evaluation. The team welcomes IEG's continued input as it considers how best to integrate GTFP's DOTS with IFC's existing DOTS process.
IFC Action 9: Execute comprehensive overview of GTFP DOTS framework
Indicator: revised framework that is operationally feasible
Target: Revised framework: structure and production process
Timeline: 3Q15
In the absence of information from the Management, IEG keeps the rating on Implementation Progress as Substantial and Implementation Status as Active (Rated). In its 2015 update, IEG acknowledged that substantial progress has been made in implementing the GTFP DOTS framework and gathering information from issuing banks in the GTFP network via surveys.
However, in its 2015 update, IEG raised a question on whether the data from surveys along with DOTS data on portfolio and finance by itself is sufficient enough to address Effectiveness at the Program-level That is, a deeper analysis of the GTFP program in selected countries would be needed in order to assess the Effectiveness of the overall GTFP program. However, IEG did not received any further response from IFC in FY15.
Furthermore, in the absence of information from IFC in FY16 and FY17, the ratings for Implementation Progress and Implementation Status remains unchanged.
In the absence of information from the Management, IEG keeps the rating on Implementation Progress as Substantial and Implementation Status as Active (Rated). In its 2015 Update, IEG acknowledged that substantial progress has been made in implementing the GTFP DOTS framework and gathering information from surveys to Issuing Banks in the GTFP network. However, in its 2015 Update, IEG raised a question on whether the data from Issuing Bank Surveys along with DOTS data on Portfolio and Finance by itself is sufficient enough to address Effectiveness at the Program Level. That is, a deeper analysis of the GTFP program in selected countries would be needed in order to assess the Effectiveness of the Program. However, IEG did not receive any further response from the Management in FY15. Furthermore, in the absence of information from Management in FY16, the ratings for Implementation Progress and Implementation Status remains unchanged.
IEG acknowledges that based on the information in the most recent management update substantial progress has been made in implementing the GTFP DOTS framework and gathering information from surveys to issuing banks in the GTFP network. As per managementâs update, progress has been made in identifying indicators that reflect GTFPâs emerging market issuing banksâ growth and contributions to trade finance. However, IEG raises question on whether data from issuing bank surveys along with DOTs data on portfolio and finance by itself sufficient to address effectiveness at the program level. That is, there needs to be deeper analysis of GTFP program in selected countries in order to attribute the program contributions to survey results.
Action 9. Significant progress has been achieved in revising the framework through also with the most recent DOTS survey (executed in February 2014) and with final reporting finalized. Furthermore, it serves as prime example of leveraging DOTS survey and results, along with other operational data to provide valuable market intel and client insight that have been absorbed, evaluated and utilized by operational staff. Further refinements to survey questions, based on operational feedback, will be completed during the next DOTS cycle. Commended by operational staff and external auditors for process efficiency and quality/QC quality as well as informative results and observations. The DOTS framework, with input directly from IEG, has focused its Development Impact analysis on the targets of development: GTFP's emerging market issuing banks. It has identified indicators to report in IFC's annual report which reflect those banks' growth and contribution to trade finance. Based on client feedback, operational staff feedback, CDI and IEG, the issuing bank survey questions have been refined to obtain development impact data that best balances value, operational feasibility, and capacity for accuracy. In addition, the survey has been updated to include voice of customer questions providing market intelligence and client intel. It combines survey responses with additional data gleaned from IFC's portfolio data and financial reports. From this information, the team is able to not only assess the development impact of GTFP, but also provide insight to operational business teams that helps them make decisions regarding their banks - both in terms of explicitly client service activities and enhancing development. It connects also client needs with our advisory service team to further enhance development. In fact, IFC's survey and the way in which information is subsequently packaged and presented to GTFP's operational teams has been commended very recently by IEG. The team that is reviewing development impact analysis has identified GTFP as one of the few programs successful in translating DOTS into specifically actionable business items.
We have just completed our individual operational team consults and review of our packages. There may be a few further survey question refinements going forward as we apply what we have learned in this year. In addition, we find that the provision of such helpful information has supported a high survey response rate - 93% - despite being the first year where all GTFP Emerging Market Issuing Banks were surveyed. We attribute this, in part, to the interest among the operational staff in getting useful results, as well as the interest in many of the banks in having a tangible venue to express their needs. The completion of this Action represents ideal collaboration between IEG, CDI and IFC Operations for a common goal.
IEG acknowledges that IFC has taken action on the GTFP DOTS Framework and reported on the results of a survey by applying this framework. However, IEG raises question about whether resulting data addresses effectiveness at the program level.
Significant progress has been achieved in revising the framework through also with the most recent DOTS survey (executed in February 2014) and with final reporting finalized. Furthermore, it serves as prime example of leveraging DOTS survey and results, along with other operational data to provide valuable market intel and client insight that have been absorbed, evaluated and utilized by operational staff. Further refinements to survey questions, based on operational feedback, will be completed during the next DOTS cycle. Commended by operational staff and external auditors for process efficiency and quality/QC quality as well as informative results and observations.