Regularly incorporating, where feasible, analytical elements, such as analysis of heterogeneous program impacts and cost-benefit analysis, in the design of all World Bank and IFC IEs.
5. Incorporating analytical elements that enhance operational relevance In the World Bank and IFC: There has been mixed coverage of analytical elements relevant for operational needs, such as analysis of distribution of program impacts; cost-benefit or cost effectiveness analysis of interventions; mapping of the causal chain from program inputs to outputs to outcomes; and measuring the contribution to impacts of individual components of program design. At the World Bank, IEs initiated in recent years appear to pay greater attention to some of these dimensions, and this trend should be sustained in future IE efforts. Similarly, these elements should be included in the design of future IFC IEs.
Original Response: IFC: The analytical elements noted by IEG will be considered by default for every impact evaluation from FY13 onward (per inclusion in the evaluation design checklist).
Specific Action: Consider specific requested IE design elements for inclusion.
Indicator: % of IEs in which all specific requested design elements were considered for inclusion, with documentation maintained by CDI.
Baseline: 0%
Target: 100% of all IEs
Timeline: FY14.
IFC developed an evaluation checklist that covers IE design elements.
It is commendable that IFC IEs include design elements. All IEs that were selected in 2014 focused on distribution of impacts including gender and social economic background. It is not clear why all focused only on distributional aspects.
100% of our current impact evaluation portfolio has considered the specific requested design elements for inclusions, and in every case have included at least one element.
IEGï¾s report listed four elements of further analysis that should be considered for inclusion in future IE work: 1. distribution of impacts/heterogeneous effects, 2. cost effectiveness, 3. data-backed mapping of the causal chain and 4. Contribution of components in program design.
IEG recognizes the efforts of CDI that allow for greater understanding of causal chain and distributional effects. However, the management does not mention why the focus was on causal chain and distributional effects. The design elements should be selected where it makes sense rather than picking the easiest to include the IE .Therefore, specific targets should be used for this recommendation. Such targets might indicate the distribution of IEs produced in the last year containing each of the four analytical elements. Also, as IFC's recent experience indicates establishment of impact and distributional aspects should be done ex-ante.
Eight of the nine IEs that launched this year--including the seven that IFC designed and considered for implementation in partnership with the WB's DIME team--all incorporate some of the delineated design elements; in particular,the mapping of the causal chain from program inputs to outputs to outcomes and/or measuring the contribution to impacts of individual components of program design. Indeed, most of these IEs are explicitly intended to test the impact of individual components of program design.
Of the three IEs that completed this year, two (DSCL Sugar and WaterHealth) made efforts to consider, ex-post, the extent to which individual components contributed to impact and the distribution of program impacts, despite the fact that this was not part of the initial evaluation design. In both cases, IFC was not able to conclusively attribute impact to specific design elements or produce meaningful distributional conclusions, which only reinforces the importance of establishing such design considerations ex-ante.
Status: Active