"Strengthening engagement and results through multiyear frameworks: Design programmatic knowledge services where appropriate in a number of thematic areas. Ensure that up to date knowledge and/or high quality research underpins recommendations and provide practical know-how and customization of global practice. Develop and apply principles to ensure balance between strategic (non reimbursable) and reimbursable activities."
"The use of programmatic approaches was important to achieve outcomes, as shown by examples of knowledge services that enabled technical assistance or economic sector work to follow up on initial work. By contrast, some projects launched in response to demand from certain agencies did not produce strong results, sometimes because these were one-off initiatives with poor sustainability prospects. Staying engaged and responsive in the implementation phases of advisory services through appropriate instruments can help clients translate recommendations from sound analysis into actions that fit local political and administrative constraints. By the same token, the focus on reimbursable advisory services to respond to client demand may entail a lack of continuous engagement in some areas or lack of coverage of thematic areas that may not rank sufficiently high on the short-term priorities of clients. The Bank's capacity to see the big picture and provide multi-sector development solutions strength of the Bank's knowledge services valued by clients may thus be eroded."
Original Response: Agreed: "WB: The Bank's knowledge work (both RAS and non- RAS) are programmed in a strategic medium-term framework defined by the CPS or CAS. CPSs/CASs are now designed to be results-based and include both lending and knowledge services. Furthermore, quality assurance processes, results measurement, and monitoring have been strengthened for both RAS and non-RAS activities, including for just-in-time advice which is increasingly demanded by our clients. It is therefore incorrect to imply that only non-RAS activities are strategic or that RAS activities are not strategic. The recommendation to develop and apply principles to ensure balance between RAS and non-RAS activities has already been implemented. Recently adopted quality assurance processes and systems have been used to strengthen the link between individual knowledge activities and the CPS/CAS results. First, teams are now required to clearly outline the objectives and intended audience, along with the intermediate outcomes that will be used to measure progress toward objectives. These objectives and outcomes are required to contribute to one of the overarching outcomes of CPSs/CASs, for those countries that have a CPS/CAS. For those countries that do not have a CPS/CAS (usually high-income countries), teams will still be required to clearly indicate the objective and audience, intended contribution to development outcomes, and related indicators of success. Second, these changes now apply also to RAS and new templates/tools have been designed for teams to ensure that the quality and strategic orientation of RASs are uniform across the Bank. Furthermore, RASs are required to have the same level of quality and follow the same processing as non-RAS ESW/TA. RAS related activities are also programmed within the CAS/CPS and guidelines have already been developed to define when a RAS should be undertaken (see the Op Memo-The Provision of Reimbursable Advisory Services available on the OPCS website). These changes help to ensure the balance between RAS versus non-RAS in the Bank's country programs. Third, at completion TTLs/task teams are required to complete a self-assessment of results achieved with greater management oversight. Fourth, this assessment is supplemented with a client feedback instrument administered to both RAS and non-RAS activities. (In fact, the CFI was just completed for the FY12 cohort and the exercise will be launched in the coming months for the FY13 cohort of completed activities). Finally, teams are encouraged to capture/document in Bank systems the evidence of the contribution of their knowledge activity to results achieved. In addition to the above, a new programmatic instrument for ESW/TA is now on the Ops Portal. Programmatic approaches are encouraged in those situations where they are relevant and appropriate, i.e., in support of medium term reform efforts."
Action 3A: Programmatic Approach (PA) guidelines, training, and reporting are being recently developed and now are being implemented.
Indicator: PA guidelines, training, and reporting implemented and adopted.
Baseline: PA guidelines, indicators in place but not implemented and adopted.
Target: Complete the implementation and adoption of PA guidelines and indicators. Develop and deliver training.
Timeline: FY16.
Judged against the limited actions identified by Bank Management for this recommendation, IEG rates it high. IEG notes that the Multiphase Programmatic Approach has been adopted and guidance is being developed and implemented.
Management would like to note that the target set for recommendation 0225 was fully completed last year. The ASA Reform introduced in FY17 improves the Programmatic Approach, with updated procedures and training already in place. The revised instrument supports longer term multi-phased client engagement model and introduces flexible scope management and program structuring.
The ASA Reform recognizes the importance of the programmatic approach and of RASs. Against the narrowly defined Action Plan, adoption is substantial. Evidence on the extent to which the ASA Reform has strengthened engagement and results, and facilitated multiyear frameworks would have been helpful. Evidence that principles have been developed and applied to ensure balance between strategic (non-reimbursable) and reimbursable activities would also have been helpful.
Management would like to note that the target set for recommendation 0225 was fully completed last year. The ASA Reform introduced in FY17 improves the Programmatic Approach, with updated procedures and training already in place. The revised instrument supports longer term multi-phased client engagement model and introduces flexible scope management and program structuring.
PA indicators developed as part of KAS, and training started. A new approach under ASA is expected to continue with this process. Management expected to implement ASA in second half of FY16. It will be important to monitor that the spirit of this objective--that PA in ASA becomes the norm rather than the exception--is observed.
As noted last year by IEG, Programmatic Approach guidelines for knowledge services were updated and posted to support new GPs and CCSAs, and training was being developed, to be launched later in calendar year 2014. During FY15, these additional actions were completed:
1) PA training was rolled out and delivered. It has reached 400+ staff (including country office staff). PA also keeps being treated in the overall AAA training, which has reached 100+ staff (including country office staff). This brings the total number of staff trained in PA to over 500 during FY 15.
2) The new ASA Dashboard has been developed and will be rolled out in October 2015. It will include indicators and management attention flags on PA to facilitate the monitoring and supervision of this product line.
Additional Information From Management: Management would like to note that the target set for recommendation 225 has been fully completed. As acknowledged in the IEG response, PA indicators have been developed and rolled out, and PA training has been fully rolled out as well. PA training has reached over 500 staff in the last fiscal year. With regards to the new ASA instrument, management can confirm that the instrument is programmatic in nature, as it will follow the spirit of IEG recommendations and the experience with the current PA instrument. That withstanding, it has to be noted that the development the new programmatic ASA instrument is beyond the scope of the action plan and should not preclude the closing of actions which have been completed. In this case, the target of "Complete the implementation and adoption of PA guidelines and indicators. Develop and deliver training" has been fully achieved and the indicator of "PA guidelines, training, and reporting implemented and adopted" can be verified.
Therefore, management is proposing the rating Complete with a status of Active (to be made inactive).
Programmatic approach guidelines for knowledge service updated and posted to support new GPs and CCSAs. Training being developed and to be launched later in calendar year 2014.
Action 3A:
What was achieved in FY14:
1) PA instructions and guidelines have been fully updated to support new GPs and CCSAs.
2) all PA processing, including for RAS (with the exception of RAS billing), has been implemented in Ops Portal.
3) Dedicated training module on PA is being developed and due to be soft launched end of October.
4) PA is treated within overall AAA training, which has reached 400+ staff (incl CO staff) in FY14.