Structured literature reviews of the Independent Evaluation Group (IEG) aim to synthesize existing research on a given topic using systematic and transparent procedures. The concept and principles of a systematic review inspire and guide the approach. A systematic review aims to identify, appraise, and synthesize all relevant research that meets explicit prespecified eligibility criteria (Higgins et al. 2019).1 Because systematic reviews are intended to be exhaustive, proper implementation can require a considerable amount of time, expertise, and resources. Snilstveit et al. (2017) report an average production time of 12–24 months (depending on the scope and resources available) for a systematic review involving a multi-person expert research team.2

Comprehensively identifying all relevant studies in large volumes of literature can be a particularly challenging and time-intensive task (Thomas, Newman, and Oliver 2013). A broad and fully systematic review often requires searching a variety of literature sources, screening many thousands of studies to identify those relevant to the review. For example, searchable databases do not always use a common set of terms or keywords to index literature. Even when a standardized nomenclature for describing an issue or topic is available, searches on websites such as Google Scholar or the World Bank eLibrary can still yield many irrelevant results. Moreover, such terms likely do not capture the full breadth or nuance of a concept perfectly (Cantrell, Booth, and Chambers, forthcoming). These factors can make it challenging to ensure that a search is comprehensive, and review teams may have to search very large volumes of literature to reach saturation of coverage for the phenomenon of interest. Many systematic reviews even fall short of the mark of conducting truly exhaustive searches (as also highlighted by Evans and Popova 2016).

IEG’s structured literature review approach falls into a subset of more rapid evidence reviews inspired by the concept of a systematic review. Other common terms used for these types of literature reviews include rapid reviews, rapid evidence reviews, and rapid evidence assessments (Littell 2018; Tricco et al. 2015). For the sake of consistency, the type of literature review evaluated in this paper will be referred to as a rapid evidence review for the remainder of the paper. The Agency for Healthcare Research and Quality categorizes such reviews according to the extent of synthesis applied to the material covered in review (AHRQ 2015). In its categorization, inventories provide a list of available evidence, along with other contextual information needed to help inform decisions related to the state of research on a given subject. However, inventories do not synthesize evidence or present conclusions related to the state of the literature. Rapid responses present the user with an answer based on the best available evidence but do not attempt to formally synthesize evidence into conclusions. Rapid reviews perform a synthesis (qualitative, quantitative, or both) to provide an answer about the direction of evidence and possibly its strength.

Though ostensibly less nuanced than a full systematic review, these approaches nonetheless abide by the same basic principles as a systematic review, such as adhering to prespecified criteria for including studies and transparently reporting on the analysis of all relevant studies identified. However, the methods used may streamline the general approach and procedures used. Many rapid evidence reviews aim to deliver results within six months or less (Ganann, Ciliska, and Thomas 2010; Snilstveit et al. 2017; Varker et al. 2015). They may also use a narrower range of search techniques or sources of literature (Haby et al. 2016; Harker and Kleijnen 2012). Such restrictions ensure that the review can be delivered within shorter timeframes and resource constraints or meet deadlines required to feed into policy and decision-making processes (Varker et al. 2015; Watt et al. 2008).

Given that structured literature reviews may be limited in their coverage of the literature and depth of analysis, it is important to understand the caveats associated with applying this approach. Narrow searches of wide-ranging topics and omitting methods of critical appraisal and synthesis may limit what can reliably be said about the state of the literature and research on a particular subject. However, the processes used to conduct structured literature reviews also vary greatly (Ganann, Ciliska, and Thomas 2010; Haby et al. 2016; Hunter et al. 2020; Varker et al. 2015). Each review inevitably establishes its own set of shortcuts and heuristics, delineating the review according to a unique set of project-specific objectives.

To reflect the variety of adjustments a rapid evidence review may adopt, it is important to treat each one as unique. Omitting certain sources of literature, search methods, and analytical approaches will affect different reviews unequally: some shortcuts might be more or less important, representing different levels of risk of bias in different reviews. For example, some sources of literature may be thematically more important regarding some research topics (as would be the case for research on health-related topics, for which searching Medline and PubMed would be intuitively more important). Alternatively, some studies may find citation and reference tracking (a method of searching discussed in “Literature Search and Analysis” in chapter 2) more important for identifying all relevant literature in some contexts (see Cooper et al. 2018; Linder et al. 2015; Papaioannou et al. 2010; and Wright, Golder, and Rodriguez-López 2014). The appropriate choice between the two can depend on the efficacy of search strategies using key terms to identify relevant literature on websites and in databases.

Furthermore, some rapid evidence reviews may also have very good coverage and depth: evidence indicates that a more thorough systematic review does not necessarily always yield different conclusions than a more abbreviated review of the same topic (for example, AHRQ 2015; Haby et al. 2016). Hence, broad-brush statements about the rigor and limitations of rapid evidence reviews do not necessarily reflect authors’ individual work or specific applications of the approach.

This chapter discussed some of the underlying concepts and terminology; the next chapter presents an example of a structured literature review based on a case study assessing the effects of the World Bank’s Doing Business report.