The Mystery of Mixing Methods
Despite significant progress on mixed methods approaches, their application continues to be (partly) shrouded in mystery, and the concept itself can be subject to misuse.
Despite significant progress on mixed methods approaches, their application continues to be (partly) shrouded in mystery, and the concept itself can be subject to misuse.
By: Jos VaessenThe lack of an explicit (and comprehensive) understanding of the principles underlying mixed methods inquiry has led to some confusion and even misuses of the concept in the international evaluation community.
For some time now mixed methods approaches have been part and parcel of mainstream debates and practices in evaluation (and other branches of the applied social sciences). Going back to at least the 1950s, the mixed methods tradition picked up momentum in the 2000s. A number of text books (e.g. Tashakkori and Teddlie (2003, 2010)) and a dedicated journal (Journal of Mixed Methods Research) are clear signs of mixed methods research becoming a research paradigm in itself.
Despite significant progress on mixed methods approaches, for many evaluation stakeholders their application continues to be (partly) shrouded in mystery. To use a simple analogy, the concept of flying is intuitively understood and appreciated by most, yet when asked to explain the underlying principles, many would fall short. This is not to say that evaluators (and researchers in general for that matter) are lost in the dark. Far from this, evaluators tend to have very sensible ideas about combining qualitative and quantitative methods.
Yet, the lack of an explicit (and comprehensive) understanding of the principles underlying mixed methods inquiry has led to some confusion and even misuses of the concept in the international evaluation community. Let me briefly highlight three manifestations of this:
Notwithstanding a continued lack of understanding as well as misuses of the term by some, the imperative for using mixed methods designs to strengthen the validity of evaluation findings is strong. IEG is continuously looking for ways to strengthen its mixed methods evaluation designs. The overall purpose of a mixed methods approach is to offset the biases and limitations of one method with the strengths of another. Greene et al. (1989) in their seminal article on the principles of mixed methods in evaluation present five more precise purposes:
The art of developing mixed methods designs goes beyond an understanding of these principles. It has become widely understood in evaluation in the field of international development that different methods have particular comparative advantages (NONIE, 2009; Bamberger et al., 2010). The ongoing challenge of bringing to bear such comparative advantages in the design and sequencing of methods in multi-level and multi-site evaluations constrained by time, budget and data considerations, will continue to require context-specific and creative solutions. In that sense, some of the mystery will always remain to challenge our thinking.
What is (good) program theory in international development? (by Jos Vaessen)
Using ‘Theories of Change’ in international development (by Jos Vaessen)
Institutionalizing Evaluation: What is the Theory of Change? (by Caroline Heider)
Influencing Change through Evaluation: What is the Theory of Change? (by Caroline Heider)