In a recently published Op-Ed in the Diplomatic Courier, I argued that while parliaments and parliamentarians continue to play a vital role in functioning democracies more than a millennium after the first such body was established, their role could be made even more effective if there was a greater embrace of evaluation.

The debates and decisions of parliamentarians should be informed by evidence from experience. If a program has produced expected results, should it continue, can it be repeated or expanded? Has a policy contributed to greater equity or heightened inequality and, if so, what corrective measures are needed? And, what better way to hold government accountable than an impartial assessment by evaluators that draws on feedback from stakeholders, including those who voted parliamentarians into office and are affected by their actions.

Yet, few countries have well established evaluation systems. And what is evaluated often relates to assistance provided by multilateral and bilateral partners who evaluate results and performance.

As the relative importance of official development assistance diminishes as a proportion of total flows for development, the reliance on these evaluations is insufficient. Instead, evaluation systems need to be able to assess the collective effects of resources for development - domestic and foreign, public and private, trade, aid and investments. Governments and parliamentarians alike should have an interest in understanding whether these monies propel development outcomes and who benefits from them.

So, if evaluation is such an important part of good governance and a crucial part of the healthy debate in the institutions of democracy, what do parliamentarians need to do?

I believe they can play a crucial role in three ways:

By promoting an evidence-based discourse in their debates. Parliamentarians should recognize the importance of using evidence to inform decisions – be they on adopting legislation, approving budgets, or debating government performance in delivering against promised development results.

By being discerning consumers of evaluation. They need to be discerning users because not all evaluations provide as strong or sound an evidence-base as they should, sometimes because their methods are flawed, but more often because the data is insufficient to be conclusive.

By safeguarding the independence of evaluation. Parliamentarians can do this by ensuring that at least part of the evaluation system is safeguarded from undue influence with reporting lines to them rather than the administration, and with independent appointments and resources.

Generating knowledge on what works, what doesn’t and why is at the heart of evaluation. Acting on that knowledge should be at the heart of governance. 

Comments

Permalink

Evaluations should, of course, play a major role in governance. But politicians have to act such that the public (or large donors) will think it's useful to act. So evaluators can help by writing reports that show, in plain language, why their results are good for the public. I mean, of course, any good evaluation will show what's good for the public, but the report has to say so, in clear, convincing language. And it helps if the public has a good understanding of what evaluation is and why it's important.

In reply to by Gene Shackman

Permalink

Gene - Caroline is away right now, so let me respond on her behalf. You make an incredibly important point -- a high quality and rigorous evaluation may have no impact if the language is not clear and stakeholders (like Parliamentarians or the public) can't easily find the information they need. This is why we are focusing more than ever on making our evaluations easier to read and digest -- stay tuned for a future post on our new "origami report" structure! Other partners - like your own Global Social Change Research Project -- play an important role too in educating evaluators and the public alike on the principles of clear evaluation communication.
Permalink

Knowledge(evaluation, evidence)----Freedom (and right of politicians to make decisions) ----Accoutnability (to public for results)

In reply to by rijad kovac

Permalink

Rijad -- very powerful summary -- and perfectly illustrates Gene Shackman's earlier point about the impact of clear and precise language. Thank you.
Permalink

The trouble is that politics is the pursuit of popularity and evidence based decision making is the pursuit of what we know to be right. What we should be doing or have done, based on the evidence, and what people want to hear are often quite different. If evidence can be used to increase popularity it's politically attractive, but if the evidence cannot, then there is little desire to use it. If I were a cynic I would therefore say that the hard and unpopular global decisions that need to be made in respect to climate change, resource consumption, global population, environmental degradation etc are not appropriate to be made within the political domain because they need to be made using the best available evidence. So how can these decisions be made?

In reply to by Rob Richards

Permalink

Rob, you are right, this is the tension between decision-makers and evaluation that we all have to deal with. While at times politics seem to make the work of evaluators harder and seemingly pointless, in this day and age of information and data revolution, providing "more than evidence" as Gene put it, is important to inform a public that can ask for different policies. Difficult, absolutely! Impossible? Hopefully not.
Permalink

Building on Gene's comment, evaluation is (of course) more than evidence alone. Evaluations will have more clout when they are clear about the evidence AND the values that are used to make evaluative judgments. That would, I think, deliver Parliamentarians information that is better aligned with their interests and needs. As Parliamentarians are not necessarily trained in either research or evaluation, it behoves evaluators not only to reach valid conclusions but to communicate them clearly, concisely, and with specific audiences in mind. That might mean more than one report is needed.

In reply to by Julian King

Permalink

Julian, thanks for bringing these points together: evidence and values, understanding stakeholder perspectives, and communicating in ways that can make a difference. As Rick mentioned, we are working hard on improving the commmunications side of our evaluations to get to this point.
Permalink

Life devoid of evaluation surely loses its value at the dawn of reality. The parliament itself is an evaluative organ meant to realign, through policy redirection, governance to its neglected essence. But, for the generality of developing democracies in sub-Saharan Africa, the tight-grip of the 'executive stakeholders' on the other two arms of government and the baseless 'exaggerated party spirit and loyalty among politicians' often rob on the potency of the Parliaments at ensuring delivery of good governance as a return on the electoral investment of the citizens. While elections, as popular verdict, could serve as a check on baseless exaggerated party loyalty, through the control of the media and election-related deployment of national finances as well as the manifest subversive maneuvering of the electoral umpires, electoral processes and outcomes sometime appears like magic than reality. Similarly, the tendency towards over-reliance on official data in evaluation often helps to conceal the stark reality of official inadequacies. For a change, I support the view that parliaments, as custodians of public conscience, should themselves, beyond four-yearly electoral sanctions, be subjected to constant mid-term evaluation, while evaluating groups such as IEG/World Bank, should rely less on official statistics but more on real-life manifestations of living among the governed.

In reply to by E.B.A. Agbaje

Permalink

Agbadje, you raise an important point about reliability of data. Some 20 years ago, this was even more a challenge. Today, we are actually tapping into a lot more than official statistics. There is Big Data that is available, often for free, and that we can sometimes explore. For instance, in one evaluation we downloaded satellite data to identify trend data for forest fires to determine deforestation. In other cases, we use social media. In one case, we hired a company to run a radio campaign that invited people to send in an sms to a phone number if they were interested in being interviewed about social services in their area. And, we increasingly use impact evaluations to systematically review them to understand better what the impacts of programs have been. Imperfect? Yes, each of these have their weaknesses, but taken together with interviews of a cross-section of stakeholders, our analysis of documentation, and the wealth of project evaluation data we have, we are getting a more and more rounded picture of what's happening on the ground. There is room for constant growth in this area, which is a good thing.
Permalink

This article aceihved exactly what I wanted it to achieve.
Permalink

Thanks Caroline for the good article and thanks commentators for other interesting points. Colleagues and I sat up upon reading this paragraph. "By safeguarding the independence of evaluation. Parliamentarians can do this by ensuring that at least part of the evaluation system is safeguarded from undue influence with reporting lines to them rather than the administration, and with independent appointments and resources." Right on! We have been advocating for many years the idea of an Evaluator General (for Canada), an officer who would report directly to Parliament. Our position is described at www.evaluatorgeneral.ca

Add new comment

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.