Evaluation and the Sustainable Development Goals: Unpacking the Issues
A structured overview of some of the key evaluative issues and questions around this topic
The principal question for many has been, 'How do we know if we are achieving the goals?'... Evaluation’s role in relation to this question is rather limited. If, by contrast, one asks the question, 'How do we get there?', then evaluation ... becomes an essential ingredient in the evidence mix.
Ever since the international community endorsed the Sustainable Development Goals (SDGs), debates have flourished on the topic. The principal question for many has been, "How do we know if we are achieving the goals?". This largely relates to the choice and definition of indicators and targets, as well as the (potential) availability of data to measure progress.
Evaluation’s role in relation to this question is rather limited. If, by contrast, one asks the question, "How do we get there?", then evaluation (the systematic inquiry into what works, why, under what circumstances, and for whom) becomes an essential ingredient in the evidence mix.
Recently, there has been a lot of interesting work on the role of evaluation in the framework of the SDGs (see, for example, the series of briefing papers on effective evaluation for the SDGs and Evaluation for Agenda 2030). As debates tend to go into different directions, there is scope for developing a structured overview of some of the key evaluative issues and questions around this topic.
The framework we present here is organized around three different levels of analysis: global level, country level and the policy intervention level.
The Sustainable Development Goals have been endorsed by UN member states across the globe. While this sends a strong signal for future action, much depends on the extent and nature of international advocacy, dialogue and awareness-raising work and how this influences the actual adoption of the SDGs by different types of institutional actors (e.g. state, private, civil society). There are a number of illustrative questions that can be asked around the effectiveness of the global SDG movement and the adoption of the SDGs by different institutional actors across countries: To what extent are international (intergovernmental) organizations advocating for the SDGs and supporting national actors to embed the SDGs in their work? To what extent are international initiatives on the SDGs effectively influencing awareness, attitudes and behaviors of key actors across countries? To what extent are sustainable development issues and their interlinkages perceived as priorities by different countries? What are the main explanatory factors?
Many of the questions listed above, i.e. those around the adoption of the SDGs by different types of institutional actors, are first and foremost to be posed at the country level (and subsequently synthesized into a global perspective). The issue of country-level adoption of the SDGs can be further unpacked into more specific questions at country level, for example: How relevant are the SDGs (and corresponding indicators and targets) at the country level? To what extent have countries integrated the three pillars of sustainable development (i.e. economic, social and environmental pillars) into national policy (e.g. including national development plans)? To what extent have countries developed coordination mechanisms for integrating SDG-related issues into program design and implementation? To what extent do these mechanisms effectively include private sector and civil society actors?
A second dimension of interest in the context of evaluating the SDGs at country level concerns the nature of the evaluation system at the country level. Relevant questions are for example: What is the state of institutionalization of evaluation? How is evaluation organized within and across sectors? What are the capacities of evaluation stakeholders (demand and supply side)? What are the opportunities and constraints for conducting cross-sectoral evaluative exercises (e.g. reflecting the interconnected nature of the SDGs)? What are the opportunities and constraints for evaluation to influence decision-making processes?
Finally, any evaluative effort that looks into the effectiveness of policy interventions in relation to the SDGs would depend on data availability at the country level on SDG-related indicators over time. Examples of relevant questions are: What is the availability of data on the indicators of the SDGs at country level? What is the periodicity of data collection? What is the quality of the data? What are the capacities of national organizations to collect relevant data? What are the opportunities and constraints for doing so?
While there are multiple dimensions of evaluative interest with regard to the SDGs at this level, we highlight two key aspects.
Dealing with causality in the context of the SDGs merits particular attention. Which interventions work, and under what circumstances, in terms of contributing to the achievement of SDG-related targets? Well-known debates on causal attribution and the application of rigorous impact evaluation methods remain as important as ever in the context of the SDGs. However, now more than ever, a causal perspective in the context of the SDGs would require paying attention to such issues as the interconnected nature of causal processes, potential trade-offs or win-win situations between two (or more) objectives of a specific intervention (e.g. poverty reduction and environmental conservation), the nature of causal processes affecting SDG-related indicators (which are more often than not non-linear, uncertain, and emergent), and the timing and sustainability of effects. Among other things, the conceptual and methodological implications of the above call for a more complexity-responsive and systemic approach to evaluation (see this blog for further discussion).
A second important evaluative dimension concerns policy coherence. Policy coherence can be conceptualized in different ways but broadly refers to the extent to which interventions complement or reinforce one another. Examples of relevant questions are: To what extent is a particular policy intervention part of a broader integrated policy framework or approach that takes into account the interconnected nature of interventions and the processes of change they may influence? To what extent is there some type of coordination between actors in relation to a particular intervention (or set of interventions)? To what extent are the efforts and objectives of a set of interventions aligned with each other or possibly working at cross purposes?
To structure our thinking on the role of evaluation in the context of the Sustainable Development Goals we need a multi-level perspective. While there are obviously interlinkages between the levels, at the same time there are particular evaluative dimensions of interest at each level. The framework presented above constitutes a useful starting point to position and connect different ongoing debates on the topic of evaluation and the SDGs. Some of these debates are more nascent than others. This is a topic for a follow-up conversation...