Back to cover

The Development Effectiveness of the Use of Doing Business Indicators

Chapter 1 | Introduction: Doing Business and Country Reforms

Highlights

Since 2003, the annual Doing Business (DB) report and indicators have aimed at providing objective measures of business regulations and their enforcement as they affect small and medium enterprises across most of the world’s economies.

DB is highly influential. It is one of the World Bank Group’s most widely read publications and offers the most used set of indicators on business regulation.

Country governments use DB in their developmental strategies and programs. The Bank Group uses DB widely, including in (i) country strategy and policy dialogue; (ii) operations (financing and advisory); and (iii) research and knowledge sharing. Researchers use DB to examine the relationship of conditions it measures to reform outcomes.

DB is highly controversial, having been the subject of multiple past reviews that revealed limitations in the indicators’ methodology, accuracy, potential biases, and the way they are used in shaping and assessing country policy reforms.

Background and Context

The Committee on Development Effectiveness of the World Bank Group Board of Executive Directors requested that the Independent Evaluation Group (IEG) carry out an independent evaluation of Doing Business (DB) to assess DB’s strategic relevance and effectiveness to client countries’ reform priorities and to the Bank Group’s strategic agenda. This evaluation considers how DB is used to guide business environment reforms—both those supported by the Bank Group and those undertaken without its support. It focuses on the influence of DB on country reforms through IEG’s mandated lens of development effectiveness. It parallels work by the Bank Group’s Group Internal Audit Vice Presidency on process and data integrity and an external expert review commissioned by the Development Economics Vice Presidency (DEC) unit focused on methodology. IEG, DEC, and the Group Internal Audit Vice Presidency shared approaches and early findings during the life span of this work. An Issues Paper reviewed existing evidence to identify key issues about the use and influence of DB indicators and their relevance to country policy reforms (World Bank 2022).

Since 2003, the annual DB report and indicators have aimed at providing objective annual measures of a set of business regulations and their enforcement across most of the world’s economies.1 From the start, DB was designed with an ambitious set of stated aims, intending to capture “the efficiency and strength of laws, regulations and institutions relevant to domestic small and medium-size companies throughout their life cycle.”2

DB indicators are presented as a guide to the legal and regulatory framework for small and medium enterprises (SMEs), intending to cover the key elements of a prototypical business life cycle from start-up to operation to exit. DB has expanded over time from 5 initial indicators covering 133 countries in DB2004 to 12 indicators (10 of which feed an overall ease of doing business [EoDB] index) reported annually and covering 190 countries in DB2020 (box 1.1). From the start, DB intended to:

  1. Mobilize demand for reform of the business regulatory environment;
  2. Inform the design of reforms (DB wanted to target “what needs to be changed” and disseminate “the experience of countries that perform well according to the indicators”; World Bank 2003a, ix);
  3. Enrich international initiatives on development effectiveness. “[R]ecognizing that aid works best in good institutional environments,” DB aimed at explicitly allowing donors to base aid on “good-quality data that can be influenced directly by policy reform” (World Bank 2003a, x); and
  4. Inform theory (DB intended to inform the field of “regulatory economics” by facilitating “tests of existing theories” and contributing to “the empirical foundation for new theoretical work on the relation between regulation and development; World Bank 2003a, x).

Influence

DB is recognized as highly influential in business regulatory reform worldwide, and it is the most used set of indicators on business regulation. Its reports are among the Bank Group’s most widely read publications,3 its website among those the most visited. Among global indicators of the business environment, it has been estimated to hold a 65 percent market share.4 As elaborated in chapter 2, many country governments use DB in their developmental strategies and programs.

The Bank Group uses DB widely, including in (i) country strategy and policy dialogue, (ii) operations (both financing and advisory), and (iii) research and global knowledge sharing. Its indicators are also widely used and analyzed in the academic literature. In addition, they are a component of many other influential indexes, including the World Economic Forum’s Global Competitiveness Index, the Heritage Foundation Index of Economic Freedom, and the Fraser Institute Economic Freedom Index. Further, DB has also been influential as an approach to indicators (box 1.1), generating spin-offs like subnational DB; Women, Business, and the Law; and Enabling the Business of Agriculture.

Box 1.1. What Are the Doing Business Indicators?

Doing Business (DB) is presented as a guide to the legal and regulatory framework for small and medium enterprises. The annual DB has expanded over time from initially reporting on 5 indicators covering 133 countries in DB2004 to reporting on 12 indicators (10 of which feed an overall ease of doing business [EoDB] index) covering 190 countries in DB2020 (figure B1.1.1).

Figure B1.1.1. Evolution of Doing Business Indicators

Image

Figure B1.1.1. Evolution of Doing Business Indicators

Source: Independent Evaluation Group.

Note: Indicators in parentheses are not included in the composite ease of doing business index. DB = Doing Business.

The indicators are intended to cover the key elements of a prototypical business life cycle from start-up to operation to exit. DB also provides an overall EoDB score and ranking by aggregating the scores for 10 areas, which it claims assesses “the absolute level of regulatory performance over time” (World Bank 2019a, 19) by “benchmark[ing] economies with respect to regulatory best practice” (World Bank 2019a, 77). The methodology collects data from informed experts who are generally intermediaries for businesses in dealing with a given area of regulation or law, such as lawyers, accountants, or freight forwarders. It uses standardized case “scenarios” to enhance country comparability. Case scenarios generally apply to a country’s largest or two largest business cities (in countries with over 100 million people, since 2015). Some cases are otherwise limited. For example, dealing with construction permits looks only at procedures associated with a warehouse of specified characteristics. The use of case scenarios and intermediary informants has contributed to the ability of DB, until DB2020, to publish 17 annual volumes covering an expanding majority of countries within a budget acceptable to the Bank Group. The indicators themselves have evolved over time, with some experiencing multiple revisions to methodology or components and the introduction of new indicators.

The DB indicators are composed of a mix of three different types of subindicators. The first type, “less is better” indicators, measure time, cost, procedures, or documents required for regulatory compliance. The second type measures the “presence of a feature,” which can be legal, regulatory, or institutional. The third type measures “outputs or outcomes” of processes that are observed in practice. Among the EoDB indexed indicators, “less is more” indicators constitute 62.5 percent of total weight in the EoDB score, “presence of a feature” indicators constitute 25.6 percent, and “output or outcome” measures constitute 11.9 percent of the score.

Source: World Bank 2009, 2010, 2011, 2012, 2013, 2014a, 2015a, 2016c, 2017b, 2018b, 2019a.

The use of the indicators is intended to link to positive development outcomes through the information and influence of the indicators on multiple key actors, leading to beneficial country reforms and development outcomes. The implicit theory of change for this evaluation reflects this hypothesized influence model (figure 1.1). Channels of influence range from direct influence on government leaders to indirect influence through donors, investors, civil society, and the press.

Country governments also use DB in their development strategies and programs. Many governments establish explicit targets for progressing in their DB ranking, and many more incorporate reforms associated with improvements in DB scores into their development plans. Some of these targets are included in the World Bank’s Country Partnership Frameworks. From DB2010 through DB2020, DB tracked 2,960 positive, country-level business regulatory reforms across 184 economies (of 190 measured). DB has explicitly encouraged competition among countries on measured reforms, celebrating top reformers.

The Bank Group uses DB widely. DB indicators are used in country strategies and policy dialogue to describe business-enabling conditions in a country, or to motivate future operations in priority areas identified in Country Partnership Frameworks, for example, or in diagnosing private sector development limitations in Country Private Sector Diagnostics (CPSDs). DB is also used sometimes to justify Bank Group lending operations or advisory services, to influence their design, to target certain projects, or to assess the progress of some projects against agreed reform objectives. For example, IEG estimates that, during the fiscal year (FY)10–20 evaluation period, about 64 percent of country strategies and 676 Bank Group projects were informed by DB indicators. Finally, the Bank Group also uses DB indicators for research and to share global knowledge.

Figure 1.1. How Doing Business Influences Actions Leading to Development Outcomes

Image

Figure 1.1. How Doing Business Influences Actions Leading to Development Outcomes

Source: Independent Evaluation Group; Doing Business website (https://www.doingbusiness.org/en/doingbusiness); review of World Bank Group literature; Kelley and Simmons 2020.

Researchers use DB to study the outcomes of regulatory design, burden, and reform. DB reports that think tanks and research organizations use the indicators “both for the development of new indexes and to produce research papers” (World Bank 2016, 21) regarding the relationship of business regulation to economic outcomes. In several areas, there are no comparable annually updated data for so many countries. Although DB’s relevance to research is hard to quantify, the DB team itself collected over 400 articles from 100 leading journals of relevance to DB, and IEG’s own structured literature review (SLR) found close to 1,900 articles of potential relevance based on search terms and a review of abstracts relevant to 10 DB indicator areas. DB2019 reported that there have been “more than 3,400 research articles discussing how regulation in the areas measured by Doing Business influence[s] economic outcomes” published in peer-reviewed academic journals, 1,360 of those published in the top 100 journals, and another 9,450 “published as working papers, books, reports, dissertations or research notes”(World Bank 2018b, 32).

Controversy

DB stands out as one of the Bank Group’s controversial undertakings. Controversies arose concerning the indicators’ methodology, accuracy, and potential biases and the way they are used in shaping and assessing country policy reforms. The Bank Group and IEG have reviewed DB several times in the past, largely to respond to such criticisms. Past reviews recommended continuation of the indicators but also identified flaws and recommended modification of their design, collection, or application (table 1.1).

Table 1.1. Findings of Past Evaluations and Reviews of Doing Business

Positive Findings

Negative Findings

Influence and use

Limitations of scope and coverage

  • “The DB indicators have motivated policy makers to discuss and consider business regulation issues. Its active dissemination in easy-to-understand language permits widespread press coverage and generates interest from businesses, nongovernmental organizations (NGOs), and senior policy makers” (World Bank 2008, xvi).
  • DB is “transformational, as it fundamentally changed the way the quality of the business environment is measured, and it catalyzed actions to address constraints” (World Bank 2016d, 8).
  • DB is “influential” and “widely used” and, although controversial, attracting “unrivaled media and high-level attention” (World Bank 2018f, 7–8).
  • With significant exceptions, DB indicators focus on the compliance costs of business regulations—time, money, or procedural steps—where less regulation is better. Exceptions include the extent of protection of property, the protection of minority shareholder rights, an index of building quality control and the range of assets that can be used as collateral. It was proposed that the Bank Group develop “separate measures (…) to capture a wider range of benefits and costs (social, economic, and environmental) if existing regulations are changed” (World Bank 2015b, xl; 2019b).
  • On paying taxes, IEG and the Independent Review Panel recommended excluding the tax rate from the indicator, stating that it was more a matter of public policy than compliance burden (World Bank 2008; Doing Business Report Independent Review Panel 2013).

Usefulness to guide or monitor reforms

Limits in tracking reform or monitoring impact

  • “Within the limits of the Doing Business indicators, most investment climate interventions produce positive intermediate outcomes in terms of improvement in time, number of procedures, and cost” (World Bank 2015b, 67).
  • IEG finds a positive link between the World Bank Group’s support for trade facilitation reform and improved Doing Business trade indicators (World Bank 2019b)
  • Doing Business indicators track the incidence of reforms but not the quality or impact of reforms. IEG recommended that the World Bank Group trace country-level impacts of DB reforms (World Bank 2008).
  • The Independent Review Panel criticized DB’s implied claim that the reforms it promotes cause economic development and growth, stating “correlations do not justify a causal interpretation” (Doing Business Report Independent Review Panel 2013, 2).

Value meriting retention

Questions on aggregation

  • The Independent Panel Review recommended that “the Doing Business report be retained as an annual flagship report” (Doing Business Report Independent Review Panel 2013, 4).
  • The ease of doing business indicators “are one of the World Bank’s most important contributions to research and public policy” (Morck and Shou 2018, 3)
  • The Independent Review Panel found that aggregating rankings across topics is challenging because it explicitly or implicitly involves a value judgment of what is “better” for doing business and how much better it is. It also found that “small revisions or inaccuracies in primary data can significantly change a country’s rankings” (Doing Business Report Independent Review Panel 2013, 3). It recommended that the World Bank publish the DB report without the overall aggregate rankings (that is, without the ease of doing business ranking).

Discontinuities

Sources: Doing Business Report Independent Review Panel 2013; Morck and Shou 2018; World Bank 2008, 2015b, 2016d, 2019b.

Note: DB = Doing Business; IEG = Independent Evaluation Group.

Evaluation Approach

To evaluate the use of DB and its relationship to country reforms, IEG first developed an Approach Paper detailing the objective, scope, and proposed evaluation methods. Several lines of inquiry were later identified in an Issues Paper (World Bank forthcoming; appendix A). This evaluation used several standard and some relatively new evaluative methods to gather information and triangulate evidence at the global, country, and project levels (appendix A). Innovations included employing supervised machine learning to support the identification of projects for the portfolio review and analysis (appendix B) and the identification of impact-related claims made within the DB reports (appendix J). Using a Bank Group DB-informed portfolio and DB-tracked reforms, the evaluation team developed statistical and econometric analyses (appendix H) to explore project and country outcomes. Because of the focus on country reforms, the team undertook 10 country case studies covering countries of diverse characteristics in terms of region, income level, fragility, level of engagement with the Bank Group, and number of DB-influenced reforms (appendix C). The team also developed five indicator area deep dives (appendix D). Due to the COVID-19 pandemic, almost all fieldwork for both case studies and deep dives was conducted virtually. The literature review included both DB’s own bibliography and a rigorous SLR (appendixes E and F). The team also reviewed country strategies (appendix G) as well as Project Performance Assessment Reports and International Finance Corporation (IFC) advisory services (AS; appendix I).

  1. Doing Business (DB) came out of a strong focus on the investment climate in the early 2000s. In a 2002 volume, World Bank chief economist Nicholas Stern stated: “The central challenge in reaping greater benefits from globalization lies in improving the investment climate—that is, in providing sound regulation of industry, including the promotion of competition; in overcoming bureaucratic delay and inefficiency; in fighting corruption; and in improving the quality of infrastructure. While the investment climate is clearly important for large, formal sector firms, it is just as important—if not more so—for small and medium-size enterprises, the informal sector, agricultural productivity, and the generation of off-farm employment. For these reasons, the investment climate is itself a key issue for poverty reduction” (Stern 2002, 52). This focus led to a strong drive for a systematic approach to diagnostics and results measurement in the area of investment climate to help “clients understand investment climate conditions in their country in comparative light and induce … action to build a sound policy and institutional environment for investment” (World Bank 2003b, 2). Even at its year of inception, DB was acknowledged to be “a cost-effective way to assess certain dimensions of the investment climate” focusing on “constraints faced by businesses (such as basic incorporation procedures)” that could be “captured effectively and inexpensively by interviewing relevant experts” (World Bank 2003b, 4).
  2. “Global Indicators Group” page of Doing Business website, under heading “Doing Business,” https://www.doingbusiness.org/en/about-us/global-indicators.
  3. Each report not only updates the national indicators but also highlights an issue of interest, in some way related to DB. Each volume reports on trends, highlights top reformers and model reforms, and cites evidence for the importance of DB overall and for the importance of the individual indicators and reform areas covered. With titles like Smarter Regulations for Small and Medium Enterprises (World Bank 2012) and Reforming to Create Jobs (World Bank 2017b), they make a case for the DB agenda.
  4. Doshi, Kelley, and Simmons 2019; Besley 2015; Haidar 2012, attributed to Harvard Berkman Center, “Media Cloud Database,” 2017.