At a recent panel discussion hosted by the Independent Evaluation Group, speakers from the World Bank Group, OECD and AidData shared their perspectives about how to strengthen data production and its use for development and policymaking. Speakers included Shanta Devarajan, Senior Director, Development Economics at the World Bank, Samantha Custer, Director of Policy Analysis at AidData, Haishan Fu, Director, Development Data Group at the World Bank, Rasmus Heltberg, Lead Evaluation Officer at the Independent Evaluation Group, and Ida McDonnell, Senior Policy Analyst at the OECD Development Co-operation Directorate.

In this Conversation series, we bring you highlights from the discussion (edited for clarity), some of which centered on IEG's recent evaluation, Data for Development.   Watch the session online

---

Brenda Barbour: What do you see as the role of data in development?

Shanta Devarajan: We think of data particularly in the policy arena as useful for decision making. You want to inform decisions or you want decisions to be better informed with data. And yet at the same time, we see in IEG’s report, there are some chronic problems we encounter time and time again with data. In every situation, I've seen, there's a problem with funding for data. Then there's also the problem about of people sharing data.

If data is so useful for informing decision makers, why do we have these problems over and over again? And then you think about the analogy with the private sector, because the private sector uses data all the time- that's how they make decisions, and these are decisions worth a lot of money. And you don’t hear complaints in the private sector that data is underfunded or unshared. I mean, they don't share it with their competitors, for sure, but within the institution, they can share it.

That makes me think that public policy decision making is very different from a decision maker looking at data about where to drill for an oil field and then taking the decision then and there. Decision-making in public policy has to be on the basis of building political consensus. Here's where data can help inform that political consensus- by putting data out there to the general public and helping to nourish the debate out there, so it's not informing the policymaker, the decision maker, it's informing the public. When the public sees the data, when they see who is benefiting from energy subsidies or they see that these children are not learning in school, they can then start [asking] "Why aren't my children learning in school?" And then the politician really has to take the decision. Whereas if left to his own devices, he may not.

This is a very different way of looking at the role of data. It's a very powerful role that data can play but it is a very different one from saying the data is there that you give to a decision maker and help him or her take a decision. But now if you accept this idea that the role that data plays is to inform the general public. And that the general public can then hold politicians accountable. That helps to explain why we have these problems of underfunding, under sharing and partnerships. Because if you think about it, if that is the role for data, then there are some governments who don't want these data to be made public.

Rasmus Heltberg: Giving voice to data users in statistical capacity projects is something that the World Bank is already doing at least in the good practice projects. To do better on data use would really require that concerted effort across task teams within the World Bank’s operations and across sectors, and that we lean heavily on country directors and country partnership frameworks to take ownership of the data agenda in each country.

Ida McDonnell: At the OECD, where I work, we have been working on the 2017 development cooperation report, which also focused on data and data for development. One of the key messages in our report is the need to mobilize more political support and more collective action to bridge what we see as a worrying data divide.

Despite all the great progress that has been made over the past decades to have more and better data, we must not forget a number of important contextual facts. Today, 44% of countries worldwide do not have adequate capacity to register births and deaths fully and that's based on the analysis of the World Bank's statistical capacity indicator. Just five countries account for 78% of all bilateral assistance related to statistics, and that's $181 million per year. Up to 2015, the World Bank was a leading investor in statistics and as we find out in the evaluation, the tide seems to be turning a little bit on that and the investments are declining.

This evaluation is extremely timely because it provides evidence of what works in terms of statistical capacity building as the World Bank has been doing, and this is the space where actually there hasn't been much evidence and evidence is quite scarce. We can learn a lot from this evaluation, and I want to stress the learning from evaluations because that's very important, because demand for data- especially related to the SDGs- has never been greater, and the pressure on countries' national statistical systems and on international organizations is very high to deliver a report on the SDGs, the indicators and we know that we actually don't have data for about two thirds of the 232 indicators.

And also at the same time, the supply of data has never been greater, and the reports and the evaluation talks about big data and the data revolution, but we know that actually about 90% of the data that's produced by big data is not even analyzed or structured. So one could argue that such a healthy supply and demand for data is an opportunity to fill the data gaps that we're worried about. This is true, at the same time, the data ecosystem is getting more and more complex with so many actors involved in producing this data and the architecture around that is very weak.

So our report argues that the data gaps risk widening further because basic capacity to seize the opportunities continues to be weak, especially so in LDCs and in fragile states. A second risk is the lack of credibility and trust in data and statistics. In this era of fake news and alternative facts, the OECD’s chief statistician misses the point - this environment of fake news and alternative facts puts even more responsibility on statisticians as the custodians of the evidence base for policy-making. Statisticians must be capable of standing up for the right of all citizens to true reliable and accessible information.

Samantha Custer: To echo what Shanta said, information is certainly never the hero. But it does play a supporting role in how leaders allocate scarce resources and accelerate development in their communities, and certainly citizens are part of that story as well. But it would be mistaken to not say that calls for a data revolution will ring hollow if we can't decode what evidence leaders use in their decision making and why. So, when we talk about the role of the World Bank in supporting evidence-based policymaking in countries around the world, I would argue that we really need to look at two dimensions of this in practice – the World Bank as a direct provider of data and evidence that people consume as well as the World Bank as a catalyst- I think Ida called it an enabler, I would agree with that aspiration. A catalyst seeking to strengthen domestic data ecosystems through technical and financial assistance.

I'll speak briefly to how the World Bank is performing vis-a-vis other development partners in both of these roles as well as reflect on how the World Bank can further increase its impact moving forward. And for this, I'll draw on a few relevant findings from a new AidData report we released just today, entitled Decoding Data Use.

In the report we analyzed the responses of 3,500 leaders from 126 low and middle income countries.  These participants come from different stakeholder groups. They answered questions about the types of data analysis they use, from what sources and for which purposes in the context of their work. They also rated the helpfulness of these sources. We analyzed the responses in an effort to offer insights to how funders, producers, advocates for data including the World Bank can best position themselves for greater impact.

In general, we found that multilateral organizations are the preferred providers of international information sources in both use and helpfulness. In fact, five out of the top 10 international sources of development data were multilateral organizations like the World Bank and the United Nations.

I will also say that the World Bank performs particularly well as a go-to source of information for nearly half of all users of international data. And remarkably, the World Bank's dominant market position is quite consistent across different types of stakeholder groups, policy areas and regions of the world. For example, leaders ranked the World Bank among the top three most used sources of data and analysis in all six policy areas that we covered in the study. But you might be asking to what extent does money matter and whether leaders choose to pay attention to information from development partners like the World Bank.

We do see that larger donors tend to attract a greater number of users of their data analysis overall. But that some organizations are clearly punching above or below their financial weight, and so according to our first value for money index, we see that multilateral organizations like the World Bank, the European Union and the International Monetary Fund are efficiently converting large development assistance budgets into greater than expected uptake of their data and analysis. Large bilateral aid providers like the US and Germany also perform well on this measure.

But financial clout is not deterministic in terms of the value that leaders derive from a particular information source. There's a difference between the use of data and the perceived helpfulness of data in policymaking. And we see that organizations that have a mandate to focus on a particular geographic region or sector such as regional development banks, the International Fund for Agricultural Development etcetera. They receive the highest marks when it comes to the perceived helpfulness of their information, and that makes sense because they are able to focus on a particular bounded or defined group of users and focus their end products to that end.

I also wanted to turn to the World Bank's more direct role or indirect role as a data catalyst through its efforts to invest in the capacity of national actors to produce high quality data and statistics. And here as the World Bank contemplates its future role in this area, I wanted to suggest that it would do well to double down in two areas of a real demand from public, private and civil society officials in the partner countries, and that is national statistics and project-level evaluations.

Haishan Fu: It is important to recognize how the development data landscape is changing – for example, if you look at the kind of data we gathered for the Millennium Development Goals, compared to what we are now looking at with the Sustainable Development Goals, we are seeing a shift away from aggregate broad trend kind of information to trying to get more integrated granular information that can really lead to localized solutions. Also, the development data community is also evolving so fast.

We are keenly aware that, if we don't acknowledge that we don't have a set map- or road map moving forward- we will be kidding ourselves and that would be total arrogance. It is precisely because we're exploring what will be new, what will be the new approach, that this discussion is so useful. It really shows that we as the World Bank need to scale up our own commitment and effort both as direct data providers, but also as an enablers.

So really in this context, to Shanta’s point, we need to explore and try to find out where we can scale up for solving development problems, and how to overcome the political barriers and political economy. Political dialogue will be very important in bringing data users and producers together at global level, but more importantly, it is critical for us to ensure the right kind of data is being used in the right place. We need to make data more broadly accessible so that we can have an informed public, and also help create the necessary political consensus that can help influence decision making.

And here, I must say that word Bank’s leadership role is at multiple levels. First of all internally I'm so glad the bank has positioned data as a top priority rather than always at the bottom of the list. Creating a World Bank Development Data Council has allowed us to pull the different parts of the institution together to influence senior management thinking and operational work in terms of where we need to do more in terms of development data priorities and to coordinate our efforts.

Both Ida and Samantha mentioned that the World Bank has played a role in creating a partnership to bring not only development donors together but also really focus on what we can do with national statistical systems. But the landscape is changing; we increasingly need to engage with those data producers, both private sector and civil societies that we have not been working with and we really have not found the best way to work with each other yet.

This is why we joined forces in creating the Global Partnership for Sustainable Data which now has more than 250 members, private sector, government and the civil societies and so on and so forth, to really see where we can work together to promote data innovation.

Barbour: Given the huge need for data, what potential funding options are available? What else could be done, beyond funding, to increase data sharing?

Devarajan: You want to make sure data is accessible to as many people as possible. But that in turn means, like all global public goods, that there is a free rider problem. No one person wants to pay the costs, and that's where you have the underfunding for data as you do for other global public goods. So if you buy this description of the role of data and the power of data and also the explanation for why it has some problems, what can we do?

The first thing is we must accept this idea that for data to be useful in bringing about the kind of change, it has to be made public, it has to be a public good. Then the second level decision is we should invest in data the way we invest in other global public goods. There's lots of talk about climate change, everyone talks about climate change as a global public good, which is true. Right, well this is on the same level as climate change. If you really think about it, we should be investing in data the way we invest in climate mitigation, in CO2 mitigation. This is as important to the future of the planet and it is going to be underfunded if we leave it to the individual countries and individual agencies to do it.

McDonnell: I'd like to focus on one aspect of data sharing which comes back again to capacity development. A key message part of our analysis is that data has focused predominantly on data production. This is because there is a large demand coming from outside of the countries to monitor global development more generally. There is a huge demand for production because the international providers are constant about the results that they achieve and they need evidence and quantified data on the results they’re achieving. And we are neglecting what we call about the whole data cycle, so we want this kind of virtuous data cycle that reinforces that demand for more and better, but also looking at data use and data sharing.

To quote Samantha and work they have done and there’s this fabulous term called Data Graveyards whereby actually the business of Development Corporation invests so much in producing data that much of it is actually not used or not shared. So, our first practical step can be and of course the question that comes to many providers as I speak about the development provider business is that they may have concerns themselves about the quality of the data and the quality of the evidence they're using to design their projects and to evaluate their projects. And that deserves its own analysis and discussion. But that already by having that principle of sharing and of transparency of data, it would really probably expose a lot of challenges and put pressure on for more harmonized approaches.

There really is a space at the country level for the donors as predominant financers of data production, but all to work better together and to also think about how you empower citizens to actually produce their own data and to monitor development from where they are.

Custer: A couple of thoughts on that. When you think about the role of the World Bank in convening this type of group, I think there's an opportunity for it to emphasize matching contributions both from international actors and country-level actors. This is something that probably Ida has picked up. The challenge with the international financing for domestic data is that it's episodic, and it's dependent upon the variable priorities that international organizations have, and when the donor has moved on, the data collection, the production stops. And you don't want that to happen. So, there is an opportunity for international contributions to kind of crowd in some sustainable funding from those local level actors.

And then the other piece, and this is something that we actually covered in the Data Graveyards Report. A very real challenge that domestic data producers have, even if they bought into the idea of transparency, is that there are many political factors that many agencies are concerned about on various levels of opening up. But even among the more enlightened that want to, they're thinking about the business model of it. How do I pay for this? That's like a dirty word in the open data community, we never talk about how do you pay for this. And so you know that would be another way to think about this convening power of a data financing mechanism, is to say how do you actually come alongside domestic data producers to deal with some of this? How do you think about their early stage infrastructure capacity that there are some costs that need to be happen, and find ways to help them actually deal with the recurring costs and to reduce the cost of that over time so that they can do this effectively.

Fu: Just to take the cue off your comment, we don't even know how much we're spending on this indeed. Up until mid-last year, the World Bank, in our own operations, we didn’t even have a way to help us tag how much of our financing to countries is related to data. We have fixed that now.

So now we're able to see that for example, over the past three years, IDA and IBRD financing for data amounted to, on average $19 million a year, but there's a huge variation depending on the country and size of the IDA or IBRD project.

When it comes to financing, we know domestic financing, lending and trust fund based support need to all come together. As IEG’s evaluation notes, we need to pursue a sustainable financing model that allows us to leverage all three streams of funding towards supporting countries. And let’s not forget what Shanta just mentioned – the idea of development data as a global public good. The issue is how we can form a global alliance of really enlightened and committed development partners to come together to pool our resources.  This idea of a global facility is something we ought to look into, despite the difficulties that so many donors are facing.

McDonnell: We've been quite preoccupied with the question of how to get this scale up in financing for statistics. We know that according to the Global Partnership for Sustainable Development Data that there’s maybe a funding gap of about $200 million to meet the minimum costs of SDG monitoring.

A key issue that we spotted through the PARIS21 work on the national strategies for statistics is that actually national statistical plans don't even have budgets. So countries don't even know how much they should be spending on statistics. And these statistical plans are quite separate to the National Development Strategies, so how can we have a more integrated approach where a cross-cutting dimension of statistics is integrated into the national development plans and is budgeted for. And then you can have the discussions with the providers and the other sources of investment around financing that.

So maybe some of that architecture should really be at the national level.

Watch the full event - Data for Development: Perspectives on the World Bank Group's Role and Contribution