Policy interventions are designed to ultimately benefit citizens, communities, institutions and society as a whole. To better understand how interventions can make a difference, for whom and under what circumstances, it is paramount that we develop useful and realistic abstractions (“theories”) of intervention realities. In a previous blog I identified four symptoms of sub-optimal use of program theory in the design and evaluation of policy interventions in international development. In this blog, I focus on the first symptom, a lack of consensus on what program theory is about. Differences in terminology, a lack of clarity on the sources of theory, and a lack of understanding of (good) theory specification all contribute to this lack of consensus.

In practice, many related terms are used when talking about program theory: theory of change, policy theory, program logic, logic model, intervention logic and logical framework are prominent examples. Although these terms are often used interchangeably, in some cases they mean different things. Funnell and Rogers (2011) present a helpful discussion in this regard.

There is also some confusion regarding the sources of program theory. One can discern three major sources. First, intervention stakeholders (donors, managers, staff, but also implementing partners, beneficiaries, etc.) all have their beliefs and expectations as to how an intervention should and will work. The other two major sources of program theory are substantive academic theory and empirical data (i.e. as a basis for grounded theory). Program theory ultimately should rely on all three sources (Chen, 1990). However, this closely relates to the purposes and uses of program theory in design and evaluation, a point that I will take up in subsequent blogs.

This brings me to the third and most important point, theory specification. To better understand the quality of theory that underlies many development interventions, I distinguish between three levels of program theory specification in ascending order of explanatory power and robustness. This framework is inspired by Toulmin’s (1958) principles for argumentation analysis and Leeuw’s (1991) use of these principles in program theory reconstruction.

To illustrate these three levels I use the example of payments for environmental services (PES), a type of intervention that has received ample support from the World Bank in collaboration with the Global Environment Facility.

The basic premise underlying PES is that different forms of land use have implications for the provision of environmental services such as biodiversity, carbon sequestration (related to climate change) or (ground) water quality and regulation. The problem is that there is usually no market for these environmental services. Consequently, private land users do not have an incentive to invest in land uses that supply them. Offering payments to private land users for the generation of these services intends to overcome this problem.

With this example in mind let us now turn to the three levels of program theory specification.

Level 1: simple successionist causation. Many international development interventions rely on this type of program theory. A simple successionist causal theory is specified as a series of causal steps of the type A leads to B. For example, payments for environmental services provided to farmers (A) leads to farmers changing their land use by introducing land use practices that are more likely to generate environmental services (B). There is usually not much detail to the theory and one is left with a rather limited perspective on causality.

Level 2: successionist causation with warrants. The warrant refers to the why part in the theory. In our example, A is expected to lead to B because adoption of the environmentally more friendly land use practice (i.e. conservation with payment for service, Figure 1) is estimated to be privately more profitable than deforestation, a clear incentive for farmers to adopt the practice. In addition, adoption of the land use practice is also profitable from a societal perspective as long as the premium paid to farmers per unit of land is lower than the costs per unit of land for continuing the more environmentally destructive land use (i.e. deforestation and use for pasture, Figure 1).

Level 3: causation with warrants and causal assumptions. The most detailed level of theory specification provides information on both the warrants and the circumstances in which it is more or less likely for B to occur as a result of A. Let us call the warrant presented above W. In our example, A leads to B, because of W, under certain circumstances (C). For example, C could refer to household-specific constraints in terms of labor or the knowledge needed to apply the land use practices. C could also refer to farmers’ beliefs and attitudes with regard to innovations. The type of causal approach that characterizes a “level 3” theory is the bread and butter of inter alia realist evaluators, who put human agency at the center of evaluative inquiry. Rather than simple successionist causation, the realist perspective relies on the principle of generative causation: what works for whom and under what circumstances (see Pawson and Tilley, 1997; Pawson, 2013). As is the case for the warrants, the causal assumptions can (and should) be vested in (social science) theory (see Vaessen and Leeuw, 2009).

While much of the World Bank’s rich analytical (theory-driven) work on PES can be characterized as “level 3”, this is not typically the case for many other program areas. Both within the World Bank Group and beyond, the bridge between analytical (theory-driven) work and operations can be rather weak and program design and evaluation continue to rely on “level 1” theories. As a result, their value in terms of offering a framework for explanation, sense-making and measurement is rather limited. Specifying the warrants and the contextual variables that relate to the expected causal steps in the program theory, and looking at existing empirical research and substantive theory for inspiration is not only good practice, it is essential for developing truly useful theories that can help us to make sense of complex intervention realities.

In subsequent blogs I will continue the discussion on using program theory in international development.

References

Chen, H.T. (1990). Theory-driven evaluations. Beverly Hills: Sage Publications.

Funnel, S. and P. Rogers (2011). Purposeful program theory: effective use of theories of change and logic models. San Francisco: Jossey-Bass.

Leeuw, F. L. (1991). Policy theories, knowledge utilization, and evaluation. Knowledge and Policy, 4, 73–92.

Pagiola, S. and G. Platais (2007). Payments for environmental services: from theory to practice. Washington D.C.: World Bank.

Pawson, R. and N. Tilley (1997). Realistic evaluation. London: Sage.

Pawson, R. (2013). The science of evaluation: a realist manifesto. London: Sage.

Toulmin, S. (1958). The uses of argument. Cambridge: Cambridge University Press.

Vaessen, J. and F. Leeuw (eds.) (2009) Mind the Gap: Perspectives on policy evaluation and the social sciences. New Brunswick: Transaction Publishers.

Have You Read?

Using ‘Theories of Change’ in international development (by Jos Vaessen)

How complicated does the (Intervention) Model have to be? (by Jos Vaessen)

Institutionalizing Evaluation: What is the Theory of Change? (by Caroline Heider)

Influencing Change through Evaluation: What is the Theory of Change? (by Caroline Heider)