The ambitious, multi-layered and interconnected agenda demands from policy-makers and practitioners that they think and act systemically, manage in and for a dynamic and complex world, and realize positive synergy while countering potential negative outcomes. At the same time, simple and pragmatic solutions are needed that work at local levels.

TWEET THIS

Last week, I traveled to the United Kingdom to take part in a special event hosted by Wilton Park, an institution that provides a platform for thinkers, policy-makers, and practitioners to discuss current issues. As one of the guest speakers, I was invited to contribute to a session on Revisiting independence, objectivity and the critically reflective role of evaluation for the SDG era.

The organizers asked us to reflect on how development is changing and what trends deserve examination; how these trends affect our institutions and evaluation functions; and what the implications are for independent evaluation. 

In responding, I covered three areas and their implications:

  1. The SDGs - “what” we are trying to achieve and for “whom”
  2. Finance for Development in the context of changing power structures and the broader landscape of actors
  3. Transformational forces including disruptive technology and volatility

1. The SDGs

There are many intricate technical issues that arise from each of the SDGs. And, there is the overall lament about the number of goals and targets. Instead of reviewing each of them, I found it more important to focus on higher level issues that stand out:

  • “No-one left behind” has become common parlance, especially in the United Nations. But, has it been accepted by all and do we know what it means in a world of greater diversity and, at the same time, seemingly uniform expectations of “what good development might look like”? Who leads the conversation and sets the norms, for instance, about the inclusion of indigenous people, their rights, and what development means for or with them?
  • The ambitious, multi-layered and interconnected agenda demands from policy-makers and practitioners that they think and act systemically, manage in and for a dynamic and complex world, and realize positive synergy while countering potential negative outcomes. At the same time, simple and pragmatic solutions are needed that work at local levels.
  • The SDGs contain inherent resource constraints and competition. Take, for example, water. SDG14 aspires to sustainable water resource management, while a number of SDGs from 2 to 6 and 9, imply greater water resource use and pollution. These constraints are often not discussed, nor is it clear how required trade-offs will be made and by whom. Greater analytical work to understand the effects of trade-offs will be needed, as much as the political economy of these decisions and their effects.

Embedded in these three issues are forceful tensions that will need to understood and balanced.

2. Finance for Development

The 2015 Addis meeting on finance for development (F4D) was timely; it preceded the SDG summit and set out an interesting agenda that clarified:

  • the need for mobilization of private capital to meet investment needs—and, I hope, to mobilize the necessary innovative powers of the private sector—which has implications for the actors involved, their development outlook and practices, as well as for demand for bankable projects;
  • the commitment to improve domestic public resource mobilization through taxation and reducing illicit financial flows, while at the same time enhancing public finance management; and
  • a recognition that Official Development Assistance is relatively speaking a much smaller portion among these three sources of finance, but that it needs to deliver on two fronts: mobilizing private capital, and influencing better development results.

3. Transformational Forces

Here, I am thinking of the many transformational forces that are at play, such as climate change, a densely interrelated financial system susceptible to shocks, and pandemics. While the world has always been a complex system, instability in one of these (or other) domains can easily transmit to the other, and often spreads from country to regions or globally. Likewise, disruptive technology is a transformational force which can bring much needed innovations, but will also force other changes. For instance, artificial intelligence will transform the nature of work and job opportunities at a time when hundreds of millions of job seekers enter the market, and many are ill-equipped for fundamentally different demands on the labor force.

These and other issues combine into formidable challenges for the development community, but also have implications for evaluation in the following ways:

First, in making strategic choices about what to evaluate, we must consider whether lessons from the past will be applicable to the future. For instance, at IEG we are completing an evaluation of carbon finance markets in spite of the fact that these markets collapsed in 2012. The World Bank Group’s history in this sector spans 17 years of work with many lessons that need to be applied as the global community moves from the Kyoto Protocol to the Paris Accord.

Second, we must innovate how we undertake evaluations, both methodologically and in terms of process:

  • Assessing inclusion/exclusion and distributional effects of interventions is essential to understand whether the world is becoming more inclusive and who benefits the most from policies and programs. Likewise, we need to develop tools to generate a deeper understanding of how societal norms and social fabric of communities and households are affected;
  • Embedding complexity in evaluation design to understand and assess synergy effects (multipliers, positive and/or negative), and a deeper interrelationship between different intervention types and their effects on behavior changes;
  • Evaluative thinking needs to be embedded in decision-processes for trade-offs and adaptation/course corrections, and evaluation needs to assess whether the right evaluative behaviors are occurring during the process of decision-making and actions;
  • Evaluation processes have to provide opportunities for learning through evaluation, which requires intellectual independence to point to blind spots or systemic issues; and
  • Evaluators need to embrace and use technology and big data in ethical ways and with consideration of weaknesses.

Third, there is need for systematic outreach, including educating discerning audiences that understand evaluations so that they can assess their quality, and know when and how to use data. This agenda is ever more important in the rise of social media and “alternative facts”.

Finally, increasing domestic public resources will also mean that more and more countries need their own evaluation capacities. A trend that has started 10-15 years ago and is continuing to gain currency. It is an area into which we as the international evaluation community have to invest.

Comments

Submitted by Joseph Schechla on Fri, 05/04/2018 - 12:44

Permalink

Many thanks for these insights. The SDGs provide encouraging opportunities to gain new perspectives on evaluation criteria, especially relevance and coherence questions. However, I am concerned about how we are selective in focusing on some new criteria, and nt others. For example the over-arching promise of the longer-term positioning of the UN Development System calls for integration of assets across the Organization and the three pillars of its Charter. However, for example, emerging SDG indicaors are mostly static, rather than guiding the measurement of prograss/change. They ignore and replace certain standing "obligations" of states with inferior "voluntary commitments." The Targets and indicators generally diminish and/or dismiss duties under treaty and peremptory norms of international law, including the existing indicators and data required to meet reporting obligations under Human Rights Treaties, for example. In other ways, the indicators do--and could better--contribute to the monitoring of those obligations. However, the indicator development has evaded the system-wide integrity, higher vantage and policy coherence promised in the longer-term positioning of the UN Development System. We propose a better approach. On the specific evaluation of Agenda 2030 performance related to land, see: SDG Land Indicators:
In Pursuit of Normative Integrity and Policy Coherence with the UN System-wide Approach at: http://www.hlrn.org/news.php?id=p2xnYw==#.WuyNYGwh01Y and PPT: http://www.hlrn.org/img/documents/Land_indicator_initiative_JS_brief.pdf.

Add new comment