Apply mechanisms for strategic identification and prioritization of IEs at the World Bank and IFC to balance learning and results measurement objectives:
At IFC, prepare and apply a strategic selection framework, in consultation with relevant stakeholders, to guide selection of IE topics of strategic relevance for results measurement, global learning, and deployment of limited IE resources.
1. Application of a strategic approach to identify IEs In IFC: Efforts to conduct IEs, mainly to supplement Advisory Services project level assessments, started in 2005 with modest resources, and 26 IEs of IFC Advisory Service operations have been completed to date. IE selection at IFC has been generally opportunistic, and not guided by a strategic framework. IEs were initiated mostly based on staff interest; results measurement team's initiatives; and availability of funding. Through a recent general evaluation strategy, approved in FY12, IFC plans to move towards a more strategic approach to identification and prioritization of evaluation opportunities, including IEs.
Original Response: IFC: The strategic plan for evaluation now in place, approved in FY12, adopts best-practice guidelines for IE selection per the World Bank Publication "Impact Evaluation in Practice" (2011), which establishes criteria for selecting evaluations depending upon what is at stake (resource use/cost, potential impact, and strategic relevance) as well as the extent to which evidence already exists vis-à-vis the issue to be evaluated, and the extent to which there is potential for learning something new and important.
Specific Action: Apply best-practice guidelines for IE selection per the WB publication "Impact Evaluation in Practice" (2011), pages 10-11, per the FY12 evaluation strategy.
Indicator: percentage of new IEs selected via best-practice guidelines, substantiated with documentation archived by CDI.
Baseline: 0%
Target: 100% of all IEs
Timeline: FY15.
IFC developed a best practice guideline in FY15 and teams are required to use it for designing and conducting new IEs. IEs are achieved in an online database. IEG considers that the action is complete.
In the previous years, IFC developed best practice guidelines. During FY15, IFC initiated 9 IEs. Five of these are active or complete. Out of five IEs, four are in SSA, three are A2F and two are IC. The IEs are bundled around certain region and areas. The resource use/cost, potential impact, and strategic relevance of these IEs are not clear. Please see the attached file for your reference.
Pilot is completed. As stated in the 2014 submission, the pilot FY15 plan was very explicit about the strategic merits of all the evaluations we considered, including the impact evaluations that we decided to select for implementation.
All evaluation resources and the Evaluation Handbook are available on the IFC website at: http://ifcintranet.ifc.org/wps/wcm/connect/dept_int_content/development…
The best-practice guidelines for IE selection per the World Bank Publication "Impact Evaluation in Practice" is targeted at impact-evaluation of social programs for policy-making decision. As IFC Investment and Advisory Services focus on cutting edge commercial products in underserved markets they are by definition innovative, scalable or the object of policy decisions. Unlike for social programs, private sector companies have well developed MIS and market research capabilities. Hence cost effectiveness considerations are much less important than client interest in the outcome of the IE. A key criterion for IFC not included in the best practices guidelines is the buy-in and feasibility with its private sector client.
Taking stock of the lessons of this pilot, IFC's Evaluation Strategy FY16-FY18 will seek to integrate IFC and IFC client needs for IE in line with the Refocused IFC and the needs of IFC clients. Private sector oriented, strategic identification and prioritization will be included in the a forthcoming IFC Evaluation Policy - a key deliverable under the FY16-FY18 Evaluation Strategy.
CDI has been selecting its IEs strategically last two years. In particular, FY15 selection sheet was comprehensive and covered different strategic priorities of IFC. However, as in the case of last year, the name of the projects, selection criteria and process were not easily accessible.
This criteria has been rated with respect to FY14; in particular, the 4 impact evaluations launched in FY14, as well as the selection process we undertook in FY14 for the FY15 plan. The 4 impact evaluations were selected after careful consideration of their strategic merits, relevant knowledge gaps, and business demand. TheFY15 plan was very explicit about the strategic merits of all the evaluations we considered, including the impact evaluations that we decided to select for implementation.
IEG welcomes IFC's strategic selection efforts. The pilot is a significant improvement and will likely affect not just relevance and results measurement of IFC IEs, but will also improve the quality, use and dissemination of IEs. The name of the projects, selection criteria and process were not available in IFC's intranet. It is not clear whether the recent selection of nine IEs, of which 4 have been undertaken, is a balanced sample of operations across regions, business lines, investment vs. advisory operations. In addition, It is not clear why IFC selected nine IEs, but proceeded with four of the IEs. Going forward, IEG suggests IFC management to continue the IE selection process in a transparent manner and make the selected IEs, and selection process easily accessible.
In FY13 thus far, IFC piloted strategic IE selection as planned. We considerednine IEs and proceeded with four. At a minimum, we adhered to the criteria set forth in Impact Evaluation in Practice, requiring consideration of resource use/cost, potential impact, strategic relevance, and the extent to which relevant knowledge already exists. In many cases, we went further and explicitly considered additional criteria, including evaluability, capacity for delivery, scope for engagement and dissemination, and budget. CDI has documentation relevant to the selection process.
Status: Active