The Delphi technique is a widely used systematic forecasting technique for achieving convergence of opinion by soliciting information from experts on a certain subject through an interactive, iterative, and anonymous approach. It was originally developed by the Rand Corporation in the 1950s, primarily for military purposes. The technique, which is rooted in the premise that “[more] heads are better than one” (Dalkey and Rourke 1972), is a group communication exercise that aims to examine a specific issue for goal setting, policy examination, and forecasting of future events (Ulschak 1983; Turoff and Hiltz 1996; Ludwig 1997). Regular surveys explore “what is”; the Delphi technique attempts to address “what could/should be” (Miller 2006)—its name is derived from the ancient Greek oracle who could predict the future. The Delphi technique typically seeks to (i) shed light on alternatives; (ii) correlate expert insight on a specific subject; (iii) provide background information for decision-making; and (iv) reveal consensus in expert opinion (Watkins, Meiers, and Visser 2012).
“The Delphi method uses informed, intuitive judgment to analyze the future.”
—Rand Corporation (original developer of technique)
The Delphi approach is different from regular surveys. It employs several rounds of data collection, usually through a questionnaire and controlled feedback to encourage reflection of one’s own contributions and those of others (Hsu and Sandford 2007). During the first round, the group of experts independently share their views with a facilitator, who reviews the data and provides a synthesis or summary. The experts review this summary and are requested to provide updated inputs through one or more additional rounds. Throughout the process, the experts have a complete record of what insights and forecasts others have shared during each round, without attribution to any specific individual, to maintain anonymity. This allows each expert to (i) review the aggregated inputs of the group; (ii) reassess their initial judgments; (iii) generate additional insights; and (iv) clarify the information developed by previous iterations (Hsu and Sandford 2007). Anonymity is an important aspect of the Delphi technique, since it ensures free expression of opinion and helps prevent potential pitfalls of conventional means of pooling group opinion, for example, reluctance to revise opinions, the influence of dominant individuals, and group pressure for conformity—either real or perceived (Dalkey and Rourke 1972). The emphasis on anonymity and confidentiality makes the Delphi technique particularly suitable for online facilitation.
The Delphi technique seeks to find convergence of opinion by asking experts to prioritize emerging issues, which enables quantitative analysis of the opinions (Thangaratinam and Redman 2005), making the process more like problem-solving. A Delphi panel continues until a significant level of consensus is reached—typically two to four rounds. Even if consensus is not reached to an adequate level, this itself can indicate that there is not significant consensus of expert opinion related to the question being asked.
The Bank Group’s Independent Evaluation Group (IEG) undertook a Delphi exercise as a part of a broader evaluation of RE. A key aim of the evaluation was to assess the institution’s readiness to help clients navigate the clean energy transition described in chapter 1. Although the objectives of the transition are clear, there is less certainty as to how the RE markets, which have experienced a considerable evolution since around 2000, will unfold. Therefore, as one element of the evaluation, IEG wanted to identify the key emerging opportunities related to RE that the institution should help clients seize and challenges on the horizon that developing countries would require help to overcome. To establish this future scenario, IEG convened a global panel of experts on RE. The goal was to apply the Delphi technique with the panel in an attempt to generate consensus of expert perspectives on the future of RE, against which the Bank Group’s readiness to help clients navigate the transition could be evaluated. The Delphi panel was one methodology in the multimethod RE evaluation to assess the Bank Group’s performance in supporting RE (see box 2.1). It is expected to inform the future direction of the institution’s support to client countries in scaling up RE to meet globally established SDGs and climate goals.
Box 2.1. Multimethod Evaluation of World Bank Group Support to Renewable Energy Development
The renewable energy (RE) evaluation was designed to assess the World Bank Group’s performance during 2000–17 in supporting clients in developing and scaling up their RE resources (World Bank 2020). Specifically, the RE evaluation attempted to answer the following questions: (i) In what ways—and how well—has the Bank Group contributed to addressing the evolving RE needs of its clients?; and (ii) What lessons from experience can be identified to strengthen the role of the Bank Group in helping clients achieve their emerging RE goals (that is, the Sustainable Development Goals and the clean energy transition)?
Source: Independent Evaluation Group.
Note: AS = advisory services; ASA = advisory services and analytics; CAS = Country Assistance Strategy; CBA = cost-benefit analysis; CPF = Country Partnership Framework; QCA = qualitative comparative analysis; PPAR = Project Performance Assessment Report; RE = renewable energy; SLR = structured literature review.
Multiple methods were applied to triangulate findings into robust conclusions. These methods included the following:
- Structured literature review. An assessment of the evolution of RE markets; a literature review of barriers to developing RE, the energy and environment outcomes of electricity produced from RE, and development impacts.
- Portfolio review and analysis. A review of 546 investment projects in the Bank Group’s RE portfolio, select World Bank advisory services and analytics and International Finance Corporation advisory services from a portfolio of 245 activities and 19 Project Performance Assessment Reports, and an in-depth review of hydropower.
- Comparative case studies. Nine purposefully selected, in-depth country case studies, and a qualitative comparative analysis of the case results to validate the theory of change and identify critical pathways to expanding RE.
- Semistructured interviews. Interviews with public and private stakeholders, development partners, and key Bank Group managers, and a survey of a purposive sample of Bank Group staff working on RE.
In addition, a global panel of experts on RE were convened to help predict emerging trends in RE:
- IEG global expert panel on RE. A structured, iterative Delphi process with a set of global experts who helped identify and prioritize emerging RE opportunities and challenges, to establish a future scenario against which the Bank Group’s capacity and position to influence can be evaluated.