Using ‘Theories of Change’ in international development

Unfortunately and unsurprisingly ... intervention design tends to be insufficiently informed by existing knowledge repositories, program theories are insufficiently articulated, and evaluators have to do a lot of digging to reconstruct the causal logic underpinning interventions

‘There is nothing so practical as a good theory’. This phrase, purportedly coined by Kurt Lewin in the 1940s and later taken up inter alia by Carol Weiss and Ray Pawson resonates well with most evaluators and development practitioners nowadays. The term theory of change is probably the most popular expression used in the international development community. Personally, I prefer the term program theory as it is less contentious. It is fair to say that in recent years countless publications, meetings and trainings have been devoted to discussing program theory in its various guises and forms. My present contribution to this ‘cacophony’ of voices is not primarily about why program theories are important and how they should be used (although I do come to these points fairly quickly), but first and foremost about what happens in practice and how this differs from what could be done to make program theory more effective in supporting accountability and learning. In this first blog of a series, I will present the framework for discussing these issues.

Program theory refers to a structured set of assumptions regarding how an intervention works (or is expected to work) and how it influences (or is expected to influence) processes of change. Most of the methodological and conceptual work regarding program theory has been developed in the realm of (ex post) evaluation. At the same time the ‘theory of change’ lingo has deeply penetrated the world of intervention design and planning, merging with existing traditions around logical frameworks and other types of results frameworks.

For the sake of argument, I will discuss the use of program theory in relation to both intervention design and evaluation. Notwithstanding the differences between these two processes, the two are closely linked. In an ideal world, evaluators build on the espoused theories of development practitioners (and related stakeholders), which are articulated during the intervention design phase and are informed by past experience and existing knowledge about what works and under what circumstances. The circle of knowledge accumulation is complete when evaluations feed into the knowledge repositories that inform intervention design. Unfortunately and unsurprisingly, intervention realities are often quite different from this ideal: intervention design tends to be insufficiently informed by existing knowledge repositories, program theories are insufficiently articulated, and evaluators have to do a lot of digging to reconstruct the causal logic underpinning interventions. There are many reasons why this is the case. Practical constraints of time, data, financial resources, staff incentives and available expertise are important explanatory factors here.

Despite these constraints, program theory has made considerable headway in the field of international development. Yet, while many development interventions as well as their evaluations nowadays boast the use of an articulated ‘theory of change’, in reality many practitioners and evaluators tend not to be overly concerned with developing a good theory (or theories). At the risk of oversimplification I highlight four ‘symptoms’ of sub-optimal use of program theory in intervention design and evaluation in international development:

Symptom 1: A lack of consensus on what program theory is about. This symptom refers to issues such as a lack of clarity on the use of terminology, the principles of theory specification, and the sources of theory.
Symptom 2: A lack of clarity on why we (should) use program theory. There are well-known intended uses of theory that most practitioners and evaluators would agree on, but which are paradoxically often not (fully) realized in practice. At the same time there are a number of potential uses of theory in evaluation that can be very insightful in our quest for developing a better understanding of our interventions, yet tend to be unknown or disregarded. The concept of rival theories and adjudication between theories is an important principle which draws into question the entrenched idea of a single ‘theory of change’. This is also closely related to a third symptom.
Symptom 3: A lack of clarity on the level of abstraction of a program theory. Trying to capture and understand large-scale interventions (and even smaller projects) -which may comprise multiple stakeholder groups, levels of intervention, sites, activities and pathways of change- in a single program theory can be rather daunting. To improve our understanding of an intervention we should consider reconstructing and testing theories at different levels of abstraction and focus. A related issue concerns the challenges arising from using intervention-centric theories.
Symptom 4: A lack of learning about new developments in theory-based assessment of policy interventions. Applications of program theory are not keeping up with new ideas, concepts and methods (both theoretical and applied) that are being developed in this area of work. This constitutes another important reason why program theory is not meeting its potential of effectively supporting evidence-based and systematic intervention design and evaluation. There have been a number of notable developments in the field of program theory that merit the attention of development practitioners and/or evaluators.

In subsequent blogs, I will focus on each symptom separately, laying out the arguments and providing illustrations and references to (recent) work on program theory where needed.

I end with a disclaimer and a note of clarification. My diagnosis, informed by my own evaluation experience in bilateral and multilateral development organizations as well as valuable insights from some of my peers, as imperfect as it is, is not intended to present any kind of final or summative judgement on the application of program theory in the field of international development. By contrast, I hope that some of the ‘symptoms’ identified above will serve the inspirational purpose of going back to the rich theoretical and empirical literature on program theory and applying some of the useful insights found therein.