We adopted a definition of service delivery that places fundamental emphasis on meeting the expectations of citizen beneficiaries.

We recently launched a series of blogs that explore the adequacy of traditional approaches to evaluation, as part of our commitment to innovation, and to developing evaluation models that adequately address complexity. IEG’s recent work on Service Delivery represents one such foray into the unknown, destined to challenge evaluation practice to better enhance development effectiveness.

Prologue

A while back, we decided to build part of our work program at IEG under three Strategic Engagement Areas to help inform choices about what to evaluate, build synergies between individual evaluations, and to enhance engagement with the World Bank Group’s management. Under the sustained services for the poor strategic engagement area, we produced working papers on Behavioral Change (we’ll blog on behavioral change soon) and Service Delivery. The service delivery paper set out to define key concepts and features, and to identify how the World Bank Group supports service delivery. It also set out to develop a framework and analytical protocol to evaluate service delivery as part of three ongoing sector evaluations (urban transport, water and sanitation, and health services).

Take off

Following extensive research into the cross disciplinary literature, as well as engagement with colleagues across various units inside the World Bank Group (such as, the Global Delivery Initiative (GDI), Governance, Social Protection, Education, Development Economics) we adopted a definition of service delivery that places fundamental emphasis on meeting the expectations of citizen beneficiaries.

To echo the World Bank Group President, Jim Kim, we defined service delivery as a series of highly localized actions by agents in public agencies or private enterprises to provide needed goods and services to citizen beneficiaries in a way that meets their expectations.

Accountability to Citizens is an Overarching Service Delivery Principle

ServiceDeliveryModel.JPG

What also became apparent is the importance of a systems approach to service delivery. We found that service delivery is much more than any given delivery mechanism, or interface, between the service deliverer and the citizen beneficiary. In fact, effective service delivery is always subject to culture, and its design spans an appreciation, as our evaluation framework identifies, of enabling conditions, ensuring appropriate service delivery inputs, and implementation mechanisms, with a view to securing relevant outputs and outcomes.

That sounds like a tall order – and it is! But, it’s also clear that no one entity, let alone single project, can be responsible for ensuring the integrity of all parts of a complex service delivery effort. That said, the framework assumes that any intervention has to be explicitly aware of its place in the delivery system, and of the extent to which all other moving parts are in motion, recognizing that a flaw in any one part of the system is likely to affect the whole. That’s just the nature of systems.

Beyond Definitions

We’re currently testing our approach on a number of fronts. First, we’re analyzing a sample of project appraisal documents that form part of the portfolios under scrutiny for the urban transport and water and sanitation evaluations. Initial findings are interesting, particularly regarding the limited extent to which all-important accountability and feedback mechanisms are evident, something that will be of interest to colleagues working on the forthcoming Citizen Engagement evaluation. We’re also planning a similar, but more developed engagement with the evaluation of health services. We’ll share more on this once we’ve fully crunched the data.

And we’re involved in an exciting learning engagement with our partners at the GDI through which we aim to further socialize our approach with colleagues across the World Bank Group that work on sectors with significant public service delivery focus such as transport, water and sanitation, education, and health and nutrition services. We’ll also share insights to feed into supports and resources made available to operational staff and mangers. Finally, we will deliver a session in May as part of Human Development Week at the World Bank Group.

So, a lot happening, and a lot to deliver! Maria Gonzalez de Asis will blog in a few weeks to fill you in on developments under the GDI, and we’ll be back later to let you know how things are going at our end. In the meantime, please share your experience with service delivery in development and/or evaluation, as well as your views on our work to date.

Read the IEG Working Paper: Introducing a Framework for Evaluating Service Delivery in Sector Evaluations: Urban Transport, Water and Sanitation, and Nutrition

Comments

Submitted by Mahina Arefin on Wed, 02/15/2017 - 05:23

Permalink

It is bold indeed. It is expected that many restriction mainly salient one will be imposed on IEG from various quarter.

Dear Mahina, Thank you for your comment. We are conscious of the exploratory nature of our efforts in this instance. As such, our learning engagement is designed to help socialize the concepts and approach within IEG, and across relevant Global Practices in partnership with colleagues at the Global Delivery initiative. We hope this will assist in promoting open dialogue through which we can all learn - we’ll let you know how things are going in later blogs.

Submitted by Tessie Catsambas on Thu, 02/16/2017 - 17:29

Permalink

How exciting that you are undertaking this analysis! I always enjoy the frank thoughts of the IEG blog, and have actually quoted it in proposals and presentations. I am ready to hear what you all learn, and let me also add two questions: (1) From a systems perspective, if we adopt an outcome mapping frame where we recognize the importance of partner-level (or institution-level) changes that are directly relevant to the overall desired outcomes for society, how would that re-frame our evaluation methodology? (2) From an SDG perspective, how far should we be willing to go to make our evaluation gender-responsive and leave no one behind?

Dear Tessie, thank you for your comment, and for your ongoing engagement with the IEG blog! It’s also great to hear you get the opportunity to put some of our thoughts to good use – thanks for the feedback. Your first question relates to the type of issue Caroline is addressing in her recent blog series. We are exploring new ways of looking at things and we’re open-minded about where that might take us. Our efforts may lead us back to basics, or they may suggest a need for adaptation in our practice. That remains to be seen – we will, of course, let you know how things are going. On your second question, we are very much advocating gender-responsiveness in planning and evaluation. In fact, our work on Service Delivery suggests the need for responsiveness to ‘citizen beneficiaries’ and the broad diversity contained therein (including, for example, gender, age, and disability) with a view to delivering effective services in an inclusive manner. Thanks again for your comment.

Submitted by Ian Davies on Sun, 02/19/2017 - 08:11

Permalink

At the risk of raining on your parade, and not to undermine your efforts, these types of "systems" or "holistic" approaches to evaluating service delivery, particularly in the spheres of public services, have been around and used for a while. It would be very interesting to find out what you are doing that hasn't been done or what are you doing differently that is innovative. Catchy title though ;)

Hi Davis, You have raised interesting and sensitive point. It will be nice to see how IEG respond. It will be recalled that WDR 2004 was on Public Service Management, PSM; in 2011 WB worked on New PSM; in 2014 there was 10 year anniversary review of PSM. Service Delivery is integral to PSM and New PSM. We participated in to 2011 event and raised serious issues of serious business demanding serious attention of WBG and IEG-WBG including need for Correct Diagnosis that is followed by Correct Prescription, Correct Surgery and Correct Recovery Management. In the Final Push to achieve MDG by 2015, we added need to find correct answer to How questions especially DOING of How and need to effectively address Complaints and Grievance issues. It is pertinent to note that existing WBG and IEG-WBG Study Reports and Conferences and Meetings Outcome Documents do not answer How questions; do not address Diagnosis, Prescription, Surgery and Recovery Management issues as One and the WBG and IEG-WBG does not have Robust Compliance and Grievance Mechanism. What can IEG do to address Information and Knowledge Gaps in this regard? It is our hope that in responding to Davis comment, the additional issues we have raised will also be addressed in the Common Interest and Common Future of IEG-WBG and WBG Internal Publics and External Publics.

Add new comment