Global indicators can be very helpful, but they also require some caution in their use. Global indicators draw our attention to problems and provide metrics to compare across economies and over time.
By: Andrew Stone
Global indicators can be very helpful, but they also require some caution in their use. Global indicators draw our attention to problems and provide metrics to compare across economies and over time. With comparative rankings and positive attention given to "top reformers" rewarded by the Doing Business (DB) team, such indicators had a strong motivating effect on many countries to pursue certain business environment reforms. A guiding principle for generators of such indicators is that "What gets measured gets done." But what if the indicators do not give the full picture or measure something that may not be a priority? In the case of DB, IEG found both in practice: the promise and the perils offered by global indicators and their use in guiding reforms.
When the Independent Evaluation Group was asked by the World Bank Group Board of Directors to evaluate the development effectiveness of how DB was being used, it quickly became evident to the evaluation team that many people valued DB for its power and potential to motivate engagement. IEG's interviews of 20 expert practitioners and managers revealed that views were overwhelmingly positive (94%) regarding DB's value to initiate a reform dialogue with a government. Yet only 60% of views were positive about DB measuring the things it claimed to be measuring, while 40% were not. Only 40% of views were positive about DB's value in identifying binding constraints, and only 20% were positive about DB's value in monitoring and evaluating reforms. Since DB was being used for all these purposes, our IEG team closely analyzed these issues.
While this analysis confirmed the strong motivating and mobilizing effects of DB for reform, it also revealed a few shortcomings in the indicators and how they were being used. One way this can occur is when DB indicators focus attention on nonbinding constraints, i.e., things that will not yield a substantial investment or supply response due to the presence of other more influential factors – for example, enacting a model regulation on deck chairs as the ship fast approaches an iceberg.
Many countries have responded to country rankings and awards with initiatives to improve their ranking: Indonesia announced its initiative to be in the top 40 countries for "ease of doing business" and Morocco the top 50. DB-informed targets and reforms fed into national development strategies. A poor ranking in DB could capture the attention of government leadership and stimulate a request to the Bank Group for an engagement. In many cases, governments created or empowered coordinating agencies explicitly focused on the DB reform agenda. Yet in several countries, IEG found serious limitations to the DB indicators and their fixed agenda in capturing binding constraints in the business environment. For example, Rwanda could not address the disadvantages of being small or landlocked through DB reforms, nor did DB capture leading constraints such as low skills (human capital), limited access to and high cost of energy, high transport and information and communication technology costs, restricted access to land, and an unlevel playing field for competition.
In countries with fragility and conflict, DB reforms could be a distraction from the main constraints to private sector-led growth. In Afghanistan, DB 2019 celebrated an advance in minority shareholder rights, which did not appear to be among the priorities of domestic businesses. In other countries, where indicators were not capturing binding constraints, the rankings offered an incomplete picture. In Morocco, which was the top-ranked North African economy in DB 2020, and Jordan, which was celebrated as a top reformer in the same edition, many local small and medium enterprises (SMEs) complained of an unlevel playing field. They reported that favored investors or state-owned enterprises got special advantages in terms of access to resources, benefits, or markets. DB is understood to have uniquely influenced the government's reform agenda in Morocco, easing procedural compliance in multiple areas, yet despite these reforms yielding strong advances in its DB ranking some fundamental and binding constraints were neither monitored nor addressed.
Furthermore, some indicators captured only a part of the problem in their area. For example, "getting credit" measured two key elements of a country's financial infrastructure: the legal rights of borrowers and lenders and the availability of credit information about firms and individuals. If access to credit was impeded by a host of other factors (including imperfections in the banking and financial sector), DB could not capture it.
Lastly, although some of the claims contained in DB reports met a high standard of evidence, most claims did not. Only a minority of those claims could be confirmed through DB's literature database or had been replicated or confirmed using a rigorous methodology. This can create a reputational risk to the Bank Group.
To generate global indicators, a certain amount of standardization and stylization is necessary. The upshot is that there may be no perfect indicator for something as complex as the "business environment," which encompasses a large constellation of policy, regulatory, and institutional factors. For this reason, some important lessons surround not how to design indicators, but how to use them.
From our evaluation, looking forward, IEG arrived at four lessons on using such global indicators:
See Also:
Add new comment