The first step towards improving education systems is assessment, is measurement. Because without having a core diagnostic of where we are, it's very hard to then have a compass, have a map as to where and how we need to improve.

Tim Johnston:

Countries around the world are facing a learning crisis and the poorest communities are being hit the hardest. The learning crisis refers to the low levels of learning outcome in primary education, particularly in low and middle-income countries. In 2022, for example, 9 out of 10 children under the age of 10 in low-income countries were learning poor. This means they couldn't read a simple text fluently in their own language. School closures due to COVID-19 made the situation even worse. High learning poverty has devastating consequences for individuals, but also for entire societies. And it's not just about access to school, but whether kids are learning the skills they need to succeed in life. The learning crisis is a complex challenge, and it requires a coordinated response. The World Bank has played a key role in sounding the alarm about the learning crisis and is also the largest external funder for education in low and middle-income countries.

How can the World Bank and other international organizations support countries in addressing this crisis?

Welcome to What Have We Learned? The Evaluation Podcast. I'm your host, Timothy Johnston, manager for corporate and human development evaluations in the World Bank's Independent Evaluation Group or IEG. In this episode, we discuss ways of confronting the crisis, drawing on lessons from the World Bank's work in basic education.

Some of these lessons are captured in a recent IEG report confronting the learning crisis and evaluation of World Bank Group support.

To help us discuss the findings of the report and how the bank and other partners are responding to the learning crisis, I'm joined today by Luis Benveniste, the global director for education at the World Bank. Luis, welcome to What Have We Learned?

Luis Benveniste:

Thank you, Tim. It's great to be here. I really look forward to our conversation.

Tim Johnston:

So, Luis, let's start with a big-picture question. What's causing learning poverty and why have countries found it so difficult to improve learning for children?

Luis Benveniste:

That's a really good question, Tim. And let us start by acknowledging that in much of the developing world, on average, a low-income country spends about $50 per child on their education. And so that is oftentimes not sufficient to cover those basic inputs that are absolutely necessary for children to learn in terms of infrastructure, in terms of learning materials, in terms of textbooks, in terms of being able to attract teachers of adequate quality. So, we need to invest more. At the same time, we need to invest better. Oftentimes, education systems are not focused or geared towards outcomes, towards learning, and they're only focusing on inputs. We know, for example, from research that while inputs are important and necessary, they're not sufficient. They need to be put to good use in order for children to learn. And so how to make education systems more efficient and more effective becomes really critical to ensure that all children can learn to their best of their abilities. There are also multiple barriers that disadvantaged children, for example, face.

On top of poverty, and geography, and disability, and gender, and ethnicity can all play a part of the ways in which some children might be left out of the education system and not be able to participate or take advantage of its opportunities. A very large number of children do not come to school ready to learn. So, they don't have those basic pre-numeracy, pre-literacy skills that are just so important in early childhood.

And so, we need to put into place interventions and actions that addresses the specific needs and the specific compounding inequalities that those children left feel and experience in order for them to learn.

Tim Johnston:

But despite all these challenges, we know that some countries have successfully improved learning outcomes. For example, in the evaluation, we highlight the examples of Vietnam, Brazil, and Kenya. So in your view, what contributed to these successes and what does that mean for the bank and other development partners to have more impact on learning?

Luis Benveniste:

I had the good fortune of starting out my career at the bank working on Vietnam, and I worked off and on for about a decade in the early 2000s where, again, I could see the transformation that took place. What is the secret sauce that, for example, countries like Vietnam put into place? I would say, first, a focus on outcomes rather than outputs. The education system oftentimes had very specific goals in mind that aligned its different parts to ensure that those outcomes were met. Secondly, a very strong political and financial commitment. The Vietnamese government had very clear commitment to providing free universal primary, and then later, secondary education for all was very clear in its policy objectives and also had a machinery in place to ensure alignment at all levels of education system towards those goals, as well as a variety of policy instruments to focus on equity in achieving those outcomes.

So, equity is being at the core of the government's policy to try to ensure that that floor was raised up and that all children will be able to achieve those minimum standards. And then what came into place was a series of scaffolding of policy reforms that trying to improve the different parts of education system. When I started working in Vietnam, we started with a primary education project that looked at foundational learning. Then, for example, we put into place a teacher education project that looked at teacher standards, teacher professional development, both in service as well as pre-service, all leading to a sustained effort and investment to strengthen the different parts of the education system with a particular goal in mind, which was to improve numeracy and literacy.

Tim Johnston:

And picking up on that, another key aspect of improving education systems is also the quality of the teachers. So, the teachers are the ones right in front of the students, engaging with them and facilitating learning. What have we learned about improving teachers? We found in the evaluation, for example, that some World Bank projects tend to focus more on the in-service teacher training, but there may be a need for a broader focus. So, what do you see going forward will be the key priorities for the bank to improve teacher careers and performance?

Luis Benveniste:

This is an area where the IEG evaluation was so useful and important because it helped us really focus in on what we've been doing, and then at the same time, shed some light into the gaps where we needed to put more effort. The in-service development was at the very core of many bank operations, and we put a lot of both analytical as well as operational tools into place to try and to support those teachers into the classroom.  

In-service training used to be characterized by cascade trainings, trainer of trainer programs, which, again, a lot of evidence was pointing towards how at each level of training that capacity was watered down and, at the end of the day, was quite ineffective.

All of that took us so far, but as the IEG report well points out, if we want to make a difference, if we want that long-term solution, we really need to look at the whole career teacher lifecycle, starting from teacher identification, teacher recruitment, teacher pre-service preparation, teacher placement, teacher professional development in service, and at the end of the day, how do we support teacher retirements and to have that cycle go on over and over again. And so, again, this is where we're putting a lot of time and energy into, both trying to look at the types of incentives are necessary to recruit better teachers into the profession, and then how to really provide a fresh thinking about what is a successful pre-service training program for teachers to be, how can we use those pre-service institutions to support the continuous development of teachers? Oftentimes in-service training was provided and focused around either separate entities or consultants that provided the training, and again, we want to create that virtuous circle between pre-service and in-service training to have a coherent, well-aligned, self-sustaining teacher development program.

Tim Johnston:

So, it's not a fast process. It takes time and it takes a lot of balance among the different functions.

Luis Benveniste:

Oftentimes we talk about the importance of systemic alignment and how we need systemic reform in education to work, but that doesn't mean that we have to do everything at the same time. The scaffolding becomes really important. So how do we pick different parts of the system to strengthen those parts, and then for them to feed into the overarching machinery, mechanisms of the education system and do their part so then we can move on to other parts and continues to have a strong, responsive education system that ultimately addresses the needs of both teachers as well as students.

Tim Johnston:

So, you really need a long-term strategic view of where you want to go and you address the things you can, but you also have a sense of where you're trying to arrive in 5 or 10 years or longer.

Luis Benveniste:

That's right.

Tim Johnston:

Yeah. Let's pick up on another point you made when you're talking about Vietnam is that that focus on the shift from just focusing on outputs to outcomes. So, in that context, what do you think are the key priorities for the bank to work with development partners to keep improving on the systems for measuring learning outcomes and also for those countries that don't yet have data on learning poverty, how do we help them close that gap and improve measurement of learning?

Luis Benveniste:

Many of the folks that are listening to us will remember the World Development Reports 2018 on learning. That was a flagship publication of the bank that put some thinking as to how do we reform and rethink education systems to ensure that kids have opportunities for real learning in the classroom.

As the World Development Report emphasized, the first step towards improving education systems is assessment, is measurement. Because without having a core diagnostic of where we are, it's very hard to then have a compass, have a map as to where and how we need to improve.

Measurement is a complex science. Building institutional capacity in our counterpart countries on how to measure, on how to assess student competencies, on how to track those competencies over the school cycle, and then ultimately then use that information to take action really also becomes a critical element of that whole cycle. Teachers need to understand where kids usually stumble, and then what it is that they as teachers can do to try to support students to trying to gain those core competencies. You need a wide variety of tools starting with measurement, and then pedagogical tools to try and to address those gaps in order to use data to as a strong force towards reversing those difficulties, those challenges that students face. We've seen many countries across the developing world being able to build that capacity and ultimately put those tools to good use, good work, and ultimately delivering results.

Tim Johnston:

Yeah. And maybe to pull a few of those things together because, on the one hand, you've spoken how education reform and improving learning is really a long-term process. You can't think in years, it really has to be built over decades. At the same time, you do want to be able to measure progress in the classroom. So one of the evaluation's findings was that only about a third of World Bank basic education projects included learning outcome as a specific objective in the project design. When I was in operations, I was in some quality review meetings where we had big debates about this, like, "Can we really show progress in learning in the span of a four or five-year project? Should we focus instead on other indicators of learning and education quality instead?" And so, I'm just wondering, from your view, is it possible to show progress on certain aspects of learning in the context of a four or five-year project? And could the bank be a bit more ambitious in terms of setting learning as an objective in its education operations?

Luis Benveniste:

Part of the challenge that task managers face is sometimes a project is only three or four years old, there's no baseline data, so you need to have baseline data to show progress. The cycles of putting this assessment exercises into place can be complex and might take time, so it may be difficult to collect achievement data over the life of a project. And so all of those issues stood in the way of really truly incorporating good measurements of learning achievement in World Bank finance projects.

The evaluation is absolutely right in that we can do better, and we should do better. And a lot of time and effort is putting into how do we facilitate this process, starting out by we are recruiting and to have on staff more psychometricians, more measurement experts in place to facilitate. These are highly technical for the data to be valuable, for the data to be truly valid. It has to be done rigorously. And so we are strengthening our own internal capacity to being able to respond to this need. At the same time, we're putting a variety of training programs to support countries into building that institutional capacity in-country for them to being able to do these exercises on their own. And then at the same time, we are also doing a very concerted effort working with other partners on how we can ultimately better coordinate and better collaborate to ensure, for example, with the UNESCO Institute of Statistics, to ensure that we have a variety of measurement tools that is more easily available and usable and ultimately comparable, that abide by international standards of measurement.

So, this then data can be reported for SDG monitoring. And in that way, we're creating global public goods that provide greater, deeper, richer information to both countries as well as to the global aid community. Teachers, school inspectors, mid-level administrators, policymakers, they can draw from the data and then use that data to reflect as to what's happening in-country, what's happening in each classroom, and being able to come up with interventions that address those needs.

Tim Johnston:

And then I'll pick up on something you mentioned earlier, which was the importance of equity and inclusion, which was also highlighted in the report. And just to speak about that a little bit more, what do you see is a particular challenge for improving learning for marginalized groups like girls, children with disabilities, ethnic minorities, or even children in conflict situations?

Luis Benveniste:

Education systems could do better in disaggregation data by groups of interest. While education statistics are pretty much universal - most countries collect basic information, it's oftentimes much harder to have good measures of disaggregation. I think that we've made a lot of progress around gender. Having gender disaggregation is pretty standard now. But for example, in the disability space, huge improvements could take place. We have very limited information to the kinds of physical, mental, emotional disabilities that children face. And so, the more granular we can get, the more customized and responsive are the types of interventions that we can put into place to address the needs of those children. So that's why it's so important having good data systems, having comprehensive mechanisms to really unpack what's happening at the classroom level, truly becomes a way to empower teachers, to empower administrators to come up with solutions to the needs that children face.

Tim Johnston:

Yeah. And those needs can be very diverse because disabilities can include a whole range of challenges for the kids, and those may require different responses.

Luis Benveniste:

An area where we're putting increasing emphasis and effort is around, for example, dyslexia, dysgraphia through what's called invisible disabilities, where if we oftentimes say that 7 out of 10 children in the developing world cannot read a paragraph by the age of 10 with comprehension, much of it, for example, is driven by dyslexia, where rough estimates are that between 15 to 20% of children are dyslexic. So that means that they have a natural challenge from the get-go on being able to learn how to read and the availability of information as to in what ways dyslexia manifests itself in different languages, in different populations is extremely limited. Being able to have a much more granular understanding of what's the incidence of dyslexia, the types of challenges that children face would help us to put into place early warning systems, early diagnostic tools that can be available for teachers to identify those students that are struggling, and then put remediation exercises for those kids to ensure that, again, they reach that level of comprehension, the level of comfort in their reading practice to being able to enjoy reading as a thing of life.

Tim Johnston:

I think we all know people that have had these challenges. And then, finally, COVID was a huge challenge for education systems, and in some places, it set back learning a year or more, and it was low-income kids were particularly hard hit. But on the other side, what are some things that we learned from COVID and are there lessons that we've drawn from the epidemic that we can use going forward to improve our response to future crises or even to improve the way kids learn today?

Luis Benveniste:

I think that some of the lessons that the pandemic gave us is, on one hand, the importance of having resilient education systems so they can continue to operate in the face of a variety of climate shocks or other shocks. And so the importance of digital platforms, of online learning, of hybrid learning, of having tools that enable teachers to being able to continue education services even when there's no face-to-face learning.

And on the other hand, how to support teachers to being able to utilize these tools for more effective teaching and learning. A lot of teachers struggled in the use of these technologies to facilitate learning. Also, the pandemic really put into evidence the ways in which learning is a social enterprise... Digital cannot replace face-to-face learning, that digital while can enhance and bring value added, that social component of learning really becomes a core part of the teaching and learning experience.

Tim Johnston:

So, thanks for joining me, Luis. It's been a fascinating conversation. To learn more about IEG's recent evaluation of the World Bank's efforts to confront the learning crisis, please visit ieg.worldbank.org. And don't forget to subscribe for future podcasts. This has been What Have We Learned? The Evaluation Podcast. Thank you for listening.