Producing evidence from the past to inform future actions serves to heighten the level and quality of public debate.

In a blog I wrote towards the end of last year, I argued that evaluation should play a greater role in the work of parliamentarians. Why? Simply put, parliamentarians engage with the World Bank Group; they play an important role debating and passing national policies and legislation with which the Bank Group interacts; they hold governments to account; and, crucially, parliamentarians are accountable to the constituencies who voted them into office.

All of these functions, as well as the informed discussions and decisions that parliamentarians engage in, require evidence – of what has worked in the past, what hasn’t and why. And it is providing just such evidence that is at the heart of evaluation. This is not to say that evidence from evaluation is the only type of information that should drive decisions. But it can, I believe, help to identify predictable risks, prevent avoidable mistakes and, ultimately, produce better results.  

What does this mean in practice?

Let’s, for example, take private sector development. IEG‘s evaluation of the Bank Group’s work in public-private partnerships (PPPs) showed that many stakeholders see these arrangements as a desirable solution to  various issues, based on the belief that the private sector is best placed to deliver services. This is often true, but not always so. Our evaluation showed that if the political will is lacking, deals will fail before negotiations are completed; or if governments do not have the capacity to negotiate, engage in, and manage PPPs, deals might not work for everyone. In the end, the delivery of services may not be as efficient as expected, and the drain on public resources may be greater than anticipated.  And, while plenty of examples exist of how service provision can be improved through PPPs, it is hard to tell whether such improvements benefit the poor.

How is this different to the advice that the World Bank Group gives?

IEG evaluates what the World Bank Group does. We provide an independent assessment of the Bank Group's activities and investment to determine what works and what needs to be improved. We are independent in that we report directly to the Executive Board rather than management, and we have a mandate to take a critical look at results and performance. The Board decides the appointment of the Director General, Evaluation, and approves the work program and budget.

IEG'€™s work is a systematic effort to ensure accountability vis-a -vis shareholders, client countries, and the public at large. So, for example, as the World Bank agrees with client countries to finance projects, or introduce policy changes, it makes commitments for which it - together with its partners - is responsible. Our role is to evaluate the extent to which commitments are met, and to provide feedback to facilitate ongoing improvement. The provision of that objective, evidence based perspective can be of significant value to parliamentarians and can be used to support debate about optimizing policy and practice.

IEG evaluations draw lessons from experience that help build on and replicate success, while preventing the repetition of avoidable mistakes. Let'€™s look at another couple of examples from private sector development.  When we evaluated the Bank Group'€™s targeted interventions to promote small and medium enterprise development, we found that too many of these interventions were not very clear about the market failures they aimed to address or about their effects on enterprise development. As a result, questions remained about whether these interventions had created more businesses, whether businesses created more jobs, or whether mechanization had produced efficiency gains and better profits? Our evaluation of interventions to improve the investment climate of client countries found relevant reforms generally produced results in terms of legislation and favorable business processes, but there was little proof that they had resulted in greater foreign investment flows. The legislation reforms also took an enterprise-centric perspective that focused less on the broader development effects of these reforms.

The fact that these important lessons might not have been learned in the absence of the evaluation process is, of course, relevant well-beyond the World Bank Group. By generating evidence of success as well as failure, evaluation can support the work of parliamentarians, helping to ensure that policy debates informed by facts from past experience result in better outcomes in the future. 

Comments

Permalink

I would like to know whether IEG has carried out evaluations on EU27 GDO's similar to that reported in the 2 following papers: [1] Aruoba, S.B.; Diebold, F.X.; Nalewaik, J.; Schorfheide, F.; Song, D. – Improving GDP Measurement: A Measurement-Error Perspective, Paper No.18954, Dec.5, 2014, National Bureau of Economic Research. [2] Kugler, P.; Jordan. T.J.; Lenz, C.; Savioz, M.R. - Measurement errors in GDP and forward-looking monetary policy: The Swiss case. –Studies of the Economic Research Centre, No 31/2004, Deutsche Bundesbank. thanking in advance Roberto VACCA
Permalink

Thank you, Caroline, for a blog that makes the link between parliamentarians and evaluation so clear. Indeed, parliamentarians can use evidence to inform and advocate for the policies they support. Yet, in many countries, parliamentarians do not even know what evaluation means. Under the EvalPartners Initiative's Enabling Environment for Evaluation Task Force, we reached out to parliamentarians to inform them about the potential role of evaluation, and invite them into an alliance with EvalPartners. The Community of Evaluators of South Asia inspired us with its success in convening a regional Parliamentarians for Evaluation group, followed by Africa, and the MENA region. In October 2014, we supported the launch of the Global Parliamentarians Forum, now renamed EvalParliament. See a 3-minute slideshow/video I created to commemorate that event: https://www.youtube.com/watch?v=nyTwV__bKFw I was impressed to see the excitement for evaluation of the members of parliament belonging to opposition parties, because evaluation would be a way that they could get more independent information and more transparency in government programs; MPs in opposition parties have limited resources to check government decisions. Also, it was exciting to see the number of women who chose to support evaluation as a means of gaining independent evidence for policymaking. I asked Hon. Kabir Hashim of Sri Lanka: "How come you support independent evaluation, when you are politicians and may sometimes choose your...'evidence' selectively?" He responded like a politician: "You see, once we have the evaluation report, we can choose what to emphasize and how to interpret it." We are grateful to Kabir Hashim who is now a minister in Sri Lanka, and the many other busy MPs, who gave their time, and have brought significant leadership to the EvalPartners coalition with parliamentarians.

In reply to by Tessie Catsambas

Permalink

Many thanks, Tessie, for sharing with everybody EvalPartners experience in working with Parliamentarians. I was equally impressed by them, their commitment to national evaluation capacity development, and the use of evidence in policy making when giving the key note at their meeting at EES in Dublin and responding to their questions.
Permalink

I must say Caroline that this blog is coming at the right time which has in a way buttressed the need for legislative researchers as well as other people who contribute to the decisions taken by parliamentarians to possess adequate evaluation skills. Speaking from experience, parliamentarians are beginning to appreciate such efforts. For instance, I work as a legislative researcher in the Macroeconomics/Public Finance Unit of the National Institute for Legislative Studies Abuja which is the research and training arm of Nigeria's national parliament, the National Assembly (it plays similar roles like the Congressional Research Services of the US Congress). Some of the Senate and House of Representatives Bills we have analyzed as well as policy/committee briefs done for parliamentarians have brought forth recommendations from the law makers. They tell us how useful such works have been in their deliberations guiding their decisions reached on national issues. Thus, the evidences generated from the evaluations done in such outputs help these parliamentarians to be much more broad based and balanced in their arguments. I wish to see more similar blogs like yours in the future which I believe will help broaden the knowledge and skill horizon of people who are involved in legislative research.

In reply to by Charles I. Obutte

Permalink

Charles, many thanks for the helpful feedback that showcases how parliamentarians have used evidence, and what it takes to get there. We’ll keep you request in mind as we continue developing our blog posts.
Permalink

In Sri Lanka the 19th amendment to the constitution was passed in the in the SL Parliament, this withholds the sole discretion of decision making and implementation of plans given by the Presidents powers and the authority of the Executive Presidency. Current President of SL Mr Paithreepala Sirisena abolished the Executive President's most high ranked position which none of the Presidents thought after coming in to power. The bill was passed during this week. However the cost of essential goods have been increased. There will be another Parliamentary election in the months or weeks to come. Hoping for a better future. Shashika Wijesinha S Sri Lanka

Add new comment

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.