Using Qualitative Comparative Analysis to Explore Causal Links for Scaling Up Investments in Renewable Energy
2 | Rationale for Qualitative Comparative Analysis in Evaluating Renewable Energy
Qualitative comparative analysis (QCA) is an analytical technique that was developed in the 1980s for use in the qualitative study of macrosocial phenomena. It has proven itself particularly useful in mixed methods research, although its introduction to evaluation is relatively recent. QCA draws on both the variable-oriented and case-oriented methodologies as a “means of bridging quantitative and qualitative analysis” (Cragun et al. 2016). It combines the use of quantitative techniques to identify patterns within one’s data with in-depth qualitative understanding of the cases and subject matter being studied. The QCA methodology uses Boolean algebra to generate a set of inferences based on underlying data across multiple qualitative cases. Thus, the methodology integrates both quantitative and qualitative analysis, generating findings that are generalizable across a wider population. QCA can help identify causal patterns when triangulated with results from other methods.
“QCA is based on two primary assumptions: change is often the result of different combinations of factors, rather than on any one individual factor; and different combinations of factors can produce similar changes.”
—Charles Ragin, who is credited with developing QCA
QCA can be a useful methodology for analyzing multiple cases in complex situations to identify causal links and explain conditions under which changes happen. QCA is especially useful for identifying and understanding cross-case patterns in small- and medium-N data sets (for example, 5–50 cases) with qualitative or mixed methods findings, or both. The application of the methodology requires an in-depth understanding of circumstances within each case and must be accompanied by a robust theory of change (ToC). When these conditions are present, the results of a QCA can be interpreted and generalized across the case studies and across the population.
The Bank Group’s Independent Evaluation Group (IEG) was carrying out a multimethod evaluation of the institution’s performance in supporting RE development in its client (emerging market) countries (box 2.1). QCA was one of the methods selected, to be applied in combination with country case studies, to (i) validate the ToC developed for the evaluation (or make adjustments as necessary), and (ii) identify different pathways through which RE can be scaled up so countries can achieve the clean energy transition. Together with other methods, QCA helped formulate key evaluative conclusions and provided the basis for recommendations that would allow the Bank Group to improve its performance in supporting its clients as they navigate an evolving global RE landscape.
Box 2.1. Multimethod Evaluation of World Bank Group Support to Renewable Energy Development
The renewable energy (RE) evaluation was designed to assess the World Bank Group’s performance during 2000–17 supporting clients in developing and scaling up their RE resources (World Bank 2020). Specifically, the RE evaluation attempted to answer the following questions: (i) In what ways—and how well—has the Bank Group contributed to addressing the evolving RE needs of its clients? and (ii) What lessons from experience can be identified to strengthen the role of the Bank Group in helping clients achieve their emerging RE goals (that is, the Sustainable Development Goals and the clean energy transition)?
Figure B2.1.1. Methods Used in the Renewable Energy Evaluation
Figure B2.1.1. Methods Used in the Renewable Energy Evaluation
Source: Independent Evaluation Group.
Note: AS = advisory services; ASA = advisory services and analytics; CAS = Country Assistance Strategy; CBA = cost-benefit analysis; CPF = Country Partnership Framework; RE = renewable energy; QCA = qualitative comparative analysis; PPAR = Project Performance Assessment Report; SLR = structured literature review.
Various methods were applied to triangulate findings into robust conclusions. These methods included the following:
- Structured literature review. An assessment of the evolution of RE markets; a literature review of barriers to developing RE, the energy and environment outcomes of electricity produced from RE, and development impacts.
- Portfolio review and analysis. A review of 546 investment projects in the Bank Group’s RE portfolio, select World Bank advisory services and analytics and International Finance Corporation advisory services from a portfolio of 245 activities, and 19 Project Performance Assessment Reports, and an in-depth review of hydropower.
- Semistructured interviews. Interviews with public and private stakeholders, development partners and key Bank Group managers, and a survey of a purposive sample of Bank Group staff working on RE.
- Independent Evaluation Group global expert panel on RE. A structured, iterative Delphi process with a set of global experts who helped identify and prioritize emerging RE opportunities and challenges.
- Comparative case studies. In-depth country case studies for a set of purposefully selected countries and a qualitative comparative analysis of the case study results to validate the theory of change for the RE evaluation and identify pathways for scaling up investments in RE to navigate the clean energy transition.
The remaining chapters focus specifically on the QCA methodology and how it was applied within IEG’s RE evaluation:
- Chapter 3 provides a road map for applying QCA,
- Chapter 4 describes how the QCA was designed including the strategic decisions made by the evaluation team, the selection of in-depth case studies, and the criteria established for scoring factors and preconditions,
- Chapter 5 analyzes the results from the QCA modeling, and
- Chapter 6 interprets findings and draws conclusions from the QCA results.