Back to cover

Using Qualitative Comparative Analysis to Explore Causal Links for Scaling Up Investments in Renewable Energy

3 | A Road Map for Applying Qualitative Comparative Analysis

Undertaking QCA requires several distinct and carefully designed steps that can further strengthen the ToC and identify potential causal (near-necessary, sufficient, or both sufficient) links within this framework, as illustrated in figure 3.1.

  1. Developing the detailed ToC. The successful application of QCA begins with the development of a detailed and robust ToC for the subject being evaluated. The ToC should include the change that is being evaluated and key factors that would drive these changes based on the theoretical and experiential understanding of the subject. A sound ToC is essential since it hypothesizes the causal relationship and underlies the interpretation of results.
  2. Identifying case studies. The qualitative data that will be used to analyze potential causal links are based on case studies. These studies will need to be sufficiently representative of the key sources of variation under scrutiny (that is, where RE expanded rapidly and where it was less prolific; where the private sector was mobilized heavily and where the public sector had a larger role). The assessments must be detailed and in-depth so that qualitative data can be accurately transposed into quantitative figures to be analyzed through QCA. It is also vital that the criteria applied across the case study assessments are clear and consistent, ensuring that the results are comparable.

    Figure 3.1. Process for Applying Qualitative Comparative Analysis

    Image
    Figure 3.1

    Source: Independent Evaluation Group.

    Figure 3.1. Process for Applying Qualitative Comparative Analysis

    Source: Independent Evaluation Group.

  3. Developing a set of explanatory factors or preconditions. Factors, or preconditions, are the causal (that is, necessary, sufficient, or both) drivers that influence outputs and outcomes. They should be developed to include the factors that influence behavior change within the ToC framework.
  4. Scoring the factors or preconditions. Once the ToC is developed and the factors or preconditions are identified and defined, as much as is feasible must be learned about each selected case. This may include the use of a combination of existing data, desk reviews, and site visits, including stakeholder interviews. The information gathered through the case studies is then analyzed to assign scores for the factors or preconditions for each case. This is a vital step in converting qualitative findings into quantitative data, and it should be consistently applied across all cases. The factors or preconditions may be coded as crisp sets (dichotomies scored as 0.0 or 1.0), fuzzy sets (with scores assigned within the 0.0–1.0 interval), or a combination of crisp and fuzzy sets.
  5. Analyzing results. At this stage, with all of the groundwork completed, the data are analyzed using specialized software.1 The QCA software is used to analyze patterns among the factors or preconditions and outputs or outcomes.2 By using Boolean algebra, the software provides a rigorous logic-based approach to identifying patterns across multiple case studies and factors or preconditions.3 In addition to confirming causal links, the QCA software also looks for combinations of factors or pathways that can lead to various solutions. A unique aspect of QCA is that it is sensitive to equifinality and will recognize the presence of multiple distinct pathways that lead to the same solution.
  6. Interpreting findings. The next step is to interpret the QCA findings. This requires referring back to the case studies and ToC to ensure that the results make sense. This stage may require seeking additional case material or revisiting the ToC, and carrying out steps 1–5 again. In this sense, the QCA can be an iterative process for seeking multiple causal pathways to address a particular problem.

Box 3.1. A Cautionary Note on Applying Qualitative Comparative Analysis

Although qualitative comparative analysis (QCA) presents a rigorous methodology for analyzing drivers of change through a sample of cases that may otherwise be too small for conventional statistical analysis, the approach has its limitations. They include the following:

  • QCA relies on the strength of the underlying theory and understanding of the subject matter that is being evaluated.
  • QCA depends on the quality, depth, and consistency of the evidence gathered through case studies.
  • The scoring of factors or preconditions can require considerable judgment and subject matter expertise, making it subjective if it is not calibrated for consistency through clear application of criteria across case studies.
  • QCA cannot cope with missing data; therefore, all factors or preconditions must be scored.
  • QCA is susceptible to varying sample size, which may reduce the diversity of data sets, especially if there are fewer than five cases and potential pathways are not found in the available data.
  • It can be tempting to apply statistical mind-sets to QCA (such as using sample size for power analysis), but with Boolean algebra and set theory as its foundation, QCA must be interpreted through a deterministic lens.

As with most complex methodologies, the weaknesses of QCA often result from the way in which it is applied (Simister and Scholz 2016).

The following three chapters discuss the way in which QCA was systematically applied in IEG’s RE evaluation. Chapter 4 discusses the strategic decisions and steps taken to prepare for running the QCA model; chapter 5 describes the QCA results and how they were analyzed; and chapter 6 documents how QCA results were interpreted and triangulated with results from other methodological sources to draw key conclusions in the context of scaling up RE.

  1. See http://grundrisse.org/qca/. .
  2. In very simple cases, patterns can be discerned visually. However, complex relationships, a large number of cases, or both require specialized computer software to identify causal patterns accurately.
  3. Unlike conventional statistics, which are based on vector algebra, the qualitative comparative analysis algorithms use Boolean algebra to identify the relationships between the pre-conditions and outputs or outcomes.
Previous chapter Next chapter