Using ‘Theories of Change’ in international development
Symptoms of sub-optimal use of program theory in the design and evaluation of policy interventions in international development
Unfortunately and unsurprisingly ... intervention design tends to be insufficiently informed by existing knowledge repositories, program theories are insufficiently articulated, and evaluators have to do a lot of digging to reconstruct the causal logic underpinning interventions
‘There is nothing so practical as a good theory’. This phrase, purportedly coined by Kurt Lewin in the 1940s and later taken up inter alia by Carol Weiss and Ray Pawson resonates well with most evaluators and development practitioners nowadays. The term theory of change is probably the most popular expression used in the international development community. Personally, I prefer the term program theory as it is less contentious. It is fair to say that in recent years countless publications, meetings and trainings have been devoted to discussing program theory in its various guises and forms. My present contribution to this ‘cacophony’ of voices is not primarily about why program theories are important and how they should be used (although I do come to these points fairly quickly), but first and foremost about what happens in practice and how this differs from what could be done to make program theory more effective in supporting accountability and learning. In this first blog of a series, I will present the framework for discussing these issues.
Program theory refers to a structured set of assumptions regarding how an intervention works (or is expected to work) and how it influences (or is expected to influence) processes of change. Most of the methodological and conceptual work regarding program theory has been developed in the realm of (ex post) evaluation. At the same time the ‘theory of change’ lingo has deeply penetrated the world of intervention design and planning, merging with existing traditions around logical frameworks and other types of results frameworks.
For the sake of argument, I will discuss the use of program theory in relation to both intervention design and evaluation. Notwithstanding the differences between these two processes, the two are closely linked. In an ideal world, evaluators build on the espoused theories of development practitioners (and related stakeholders), which are articulated during the intervention design phase and are informed by past experience and existing knowledge about what works and under what circumstances. The circle of knowledge accumulation is complete when evaluations feed into the knowledge repositories that inform intervention design. Unfortunately and unsurprisingly, intervention realities are often quite different from this ideal: intervention design tends to be insufficiently informed by existing knowledge repositories, program theories are insufficiently articulated, and evaluators have to do a lot of digging to reconstruct the causal logic underpinning interventions. There are many reasons why this is the case. Practical constraints of time, data, financial resources, staff incentives and available expertise are important explanatory factors here.
Despite these constraints, program theory has made considerable headway in the field of international development. Yet, while many development interventions as well as their evaluations nowadays boast the use of an articulated ‘theory of change’, in reality many practitioners and evaluators tend not to be overly concerned with developing a good theory (or theories). At the risk of oversimplification I highlight four ‘symptoms’ of sub-optimal use of program theory in intervention design and evaluation in international development:
In subsequent blogs, I will focus on each symptom separately, laying out the arguments and providing illustrations and references to (recent) work on program theory where needed.
I end with a disclaimer and a note of clarification. My diagnosis, informed by my own evaluation experience in bilateral and multilateral development organizations as well as valuable insights from some of my peers, as imperfect as it is, is not intended to present any kind of final or summative judgement on the application of program theory in the field of international development. By contrast, I hope that some of the ‘symptoms’ identified above will serve the inspirational purpose of going back to the rich theoretical and empirical literature on program theory and applying some of the useful insights found therein.