Search

Topic:Evaluation Capacity Development
Displaying 1 - 10 of 179

Investing in Evaluation Capacity Development in India: Why it Matters Now More than Ever

Web Resource
Investing in Evaluation Capacity Development in India: Why it Matters Now More than Ever
Download the infographic Download the infographic

When evaluators cannot make it to the field, they can always observe from space

Web Resource
change in forest cover of the land surface in Madagascar from 1990 to 2017.
Field missions are at the very core of project evaluation. An evaluator will start with a desk-based review of available project information and prepare a methodology to assess the effectiveness of a project. However, it is only by interacting with policymakers, implementing agencies, and project beneficiaries that the evaluator gets a better understanding of the reality affecting the design and Show MoreField missions are at the very core of project evaluation. An evaluator will start with a desk-based review of available project information and prepare a methodology to assess the effectiveness of a project. However, it is only by interacting with policymakers, implementing agencies, and project beneficiaries that the evaluator gets a better understanding of the reality affecting the design and implementation of projects. This ‘reality-check’ stimulates learning and allows the evaluator to fine-tune their questions and methodology. The current COVID-19 travel restrictions pose significant challenges to field-based assessments of project effectiveness. So, what can evaluators do when they can’t get in the field? One possibility is to observe project impacts from space. Geospatial data is information collected by satellites pinpointed to an exact geographical location on earth. It is often freely available, covers several time periods, and offers a wide range of interesting indicators. Popular geospatial data are indicators of market accessibility, agroecology, and the environment. A geospatial dataset can thus be constructed by linking multiple geospatial data points with the geographical location of project activities and their surroundings. The possibility to construct a geospatial dataset for evaluating a project provides a unique opportunity for a robust quantitative assessment of project effectiveness. Beyond effectiveness, geospatial data can also provide a wealth of descriptive information that allows evaluators to better understand the local context. Even if visiting a project site is no longer possible due to COVID-19 related travel restrictions, evaluators can get a detailed picture of what is happening where in the project area by observing from space. IEG is analyzing geospatial datasets in several of its ongoing evaluations including an urban transport project in Mozambique, a sustainable land- and water-management project in Ethiopia, and a biodiversity project in Madagascar. Geospatial analysis usually involves two steps. First, geospatial data is used to precisely and accurately measure an indicator of project effectiveness. When a chronological series of geospatial data is available, changes in the indicator can be calculated using different measurements over time. The geospatial data on land use and road infrastructure are of particular interest to IEG’s evaluations. The ‘vegetation greenness’ of the land in Ethiopia is measured by looking at changes over time in the coverage of land with green vegetation. Similarly, deforestation rates in Madagascar are measured as the change in forest coverage of the land surface over time. In Mozambique and India, the density of social and economic activities is measured by the travel distance to urban amenities using roads. Second, as geospatial information is available for locations beyond the project boundaries, a proper ‘counterfactual’ can be constructed. The counterfactual illustrates a ‘with and without’ scenario - what would have happened at the project location if project activities were not implemented there. Combining the temporal and spatial variation in geospatial data provides a very robust ‘difference-in-difference’ assessment of project effectiveness. The temporal variation identifies the ‘before-and-after’ difference, and the spatial variation identifies the ‘with-and-without’ difference. The ‘difference-in-difference’ assessment of project effectiveness is applied as follows. In Mozambique and India, IEG compares changes in economic activity between urban areas that were either adjacent to a road improved by the project or adjacent to a nearby but non-improved road. Similarly, long-term changes in vegetation cover in Ethiopia are compared between land parcels in treated watersheds with similar parcels in untreated watersheds within a reasonable distance from the project site. Finally, IEG compares changes in deforestation rates between patches of forests on either side of the border of conservation areas in Madagascar. Then, these changes are compared between conservation areas supported by the World Bank and areas without project support. In each of these scenarios, the analysis informs the broader question of ‘what difference did the project make?’.      This 3D map shows the changes in the height of the built-up area in Mumbai. However, not all projects allow for a geospatial analysis of effectiveness. The availability of geospatial data to measure project indicators depends on the sector, the type of project, and the nature of activities. Projects without a specific geographic location, such as projects supporting a development policy at the national level, do not lend themselves to a geospatial analysis. But even if a quantitative geospatial analysis is possible, asking whether a project was effective might not be the most important question for the project evaluation. The more interesting evaluation questions are often those looking at the factors limiting the project’s impact. These factors are often highly contextual and linked with human behavior, which is much more difficult to measure from space. So, the quantitative geospatial analysis is an important first step to assess project effectiveness, but evaluations need to go further and understand why the project has been effective or not. But geospatial data can have an important contribution here as well. The geospatial information on contextual factors, such as the cover of the land or travel time to reach a given location, can help to identify different levels of project effectiveness and understand the role of underlying drivers in explaining the observed differences. In a follow-up blog, we will elaborate on how geospatial analysis can guide the design of a qualitative data collection method. Pictured at the top of the page: This image displays the change in forest cover of the land surface in Madagascar from 1990 to 2017. The black line represents the boundary of two Protected Areas for biodiversity conservation in Madagascar, Manongarivo and Tsaratanana. The blue dotted line is the 5 km buffer around the border of the Protected Area. Green dots are land that remained forest over time, white dots are land covered with forests, and red dots are land that were deforested during the period 1990 to 2017. IEG analyzed the share of different dots on each side of the border of Protected Areas to assess deforestation rates in and around Protected Areas.

Meet the Evaluator: Lauren Kelly

Web Resource
Meet the Evaluator: Lauren Kelly
The Independent Evaluation Group’s Lauren Kelly speaks on the role of the evaluator during the ongoing pandemic – and infodemic. The Independent Evaluation Group’s Lauren Kelly speaks on the role of the evaluator during the ongoing pandemic – and infodemic.

Building the First-Ever Partnership Focused on Addressing Global Gaps in Evaluation Capacity

Web Resource
Building the First-Ever Partnership Focused on Addressing Global Gaps in Evaluation Capacity
With the ten-year countdown to the Sustainable Development Goals (SDGs) now underway, and countries across the globe struggling to cope with the COVID-19 pandemic, the capacity to gather data to inform decisions, and to monitor and evaluate the impact of policies, is now an urgent priority. A broad coalition of governments and national and international organizations have agreed to establish the Show MoreWith the ten-year countdown to the Sustainable Development Goals (SDGs) now underway, and countries across the globe struggling to cope with the COVID-19 pandemic, the capacity to gather data to inform decisions, and to monitor and evaluate the impact of policies, is now an urgent priority. A broad coalition of governments and national and international organizations have agreed to establish the first-ever global partnership focused on addressing the worldwide gaps in monitoring and evaluation capacity. In June 2020, a range of donor countries and organizations met for the first time in a Co-Creation Workshop to discuss concrete steps towards establishing an inclusive partnership to meet the global demand from developing countries for stronger monitoring and evaluation systems and capacity. The aim of the partnership is to increase coordination for greater impact among the various national and international initiatives aimed at building evaluation capacity, and to pool resources and draw on local and global expertise and knowledge to scale up these efforts. The workshop was a three-day virtual brainstorming discussion focused on building consensus around a joint vision of the global partnership. Ahead of the workshop, a series of consultations were held with representatives from countries committed to strengthening their M&E systems and capacities, to understand the challenges they face, and how best to support their programs. {"preview_thumbnail":"/sites/default/files/Data/styles/video_embed_wysiwyg_preview/public/video_thumbnails/ftcYucMsIlE.jpg?itok=y3Fn6dDT","video_url":"https://youtu.be/ftcYucMsIlE","settings":{"responsive":0,"width":"854","height":"480","autoplay":0},"settings_summary":["Embedded Video (854x480)."]} Effective monitoring and evaluation systems are an essential ingredient for advancing the sustainable development goals as they foster accountability and evidence-based policy making. This innovative partnership will take us a step closer in addressing the worldwide demand from countries for stronger M&E systems and capacities for more inclusive and sustainable development results. Oscar A. Garcia, Director, Independent Evaluation Office, UNDP The current demand for evaluation capacity development far outstrips the resources and reach of any single institution, and the impact of the many programs launched to meet this need is diluted by a lack of coordination. Earlier this year, IEG and the UNDP’s Independent Evaluation Office (IEO) signed an agreement on closer collaboration on meeting this need. Recently, IEG also signed an agreement  with Canada’s École nationale d'administration publique (ENAP) in order to coordinate actions and pool expertise and resources towards meeting the need for stronger M&E systems and capacity in key, under-served regions of the world. The lack of robust monitoring and evaluation systems leaves many countries at a disadvantage and has become an ever more urgent development challenge in the face of the fast-moving coronavirus pandemic. Only by working together will we be able to address the global gaps in evaluation capacity, and ensure no communities or countries are left behind. Alison Evans, World Bank Vice President and IEG Director-General Along with developing a joint vision, the participants in the Co-Creation Workshop also discussed the key lines of business and activities in providing countries support on strengthening their monitoring and evaluation systems and capacities. They also discussed other important aspects of the partnership, such as its operational principles, budgetary and administrative arrangements, and its governance structure. The workshop concluded with an agreement amongst the participants on key steps and actions they will be taking in collaboration with the co-hosts of the workshop - the World Bank’s Independent Evaluation Group (IEG) and the Independent Evaluation Office (IEO) of United Nations Development Programme (UNDP)  - and other partners towards launching the partnership later this year.  Watch and hear from Wilson Braganca, the Director General of the Ministry of Planning and Finances from Sao Tome to learn more about the global demand for evaluation capacity development from countries. Read more about the current challenges in global M&E capacity and the need for joint action. Sign up to receive updates about the growing global partnership to close the gap in monitoring and evaluation capacity worldwide.

Independent Evaluation Group and École nationale d'administration publique formalize collaboration towards building a new global partnership

Web Resource
Independent Evaluation Group and École nationale d'administration publique formalize collaboration towards building a new global partnership
The World Bank's Independent Evaluation Group (IEG) and the École nationale d'administration publique (ENAP) have signed a Memorandum of Understanding (MOU) to pool resources and expertise to meet the global demand for stronger monitoring and evaluation capacity.  Working together, they will be in a much stronger position to meet this demand in key regions, including Francophone Africa, Show MoreThe World Bank's Independent Evaluation Group (IEG) and the École nationale d'administration publique (ENAP) have signed a Memorandum of Understanding (MOU) to pool resources and expertise to meet the global demand for stronger monitoring and evaluation capacity.  Working together, they will be in a much stronger position to meet this demand in key regions, including Francophone Africa, North America and the Caribbean, and the Middle East and North African regions.   The agreement comes at a time when there is a great need for evaluation capacity building around the world, which no single organization can meet on its own. Formalizing the collaboration between IEG and ENAP will enable the two institutions to work together to share knowledge and lessons learned, coordinated their efforts and expertise, and capitalize on their respective networks to advance evaluation capacity development in key areas of the world. The collaboration will help expand the delivery of ENAP's Programme International de formation en évaluation du développement (PIFED) to geographic and linguistic spaces that remain currently under-served. Watch IEG Director-General Alison Evans and ENAP Director-General Guy LaForest introduce this new collaboration for global evaluation capacity development. {"preview_thumbnail":"/sites/default/files/Data/styles/video_embed_wysiwyg_preview/public/video_thumbnails/oi7W5IK2YkQ.jpg?itok=GtdFWLJh","video_url":"https://youtu.be/oi7W5IK2YkQ","settings":{"responsive":0,"width":"854","height":"480","autoplay":0},"settings_summary":["Embedded Video (854x480)."]}  "This agreement will allow ENAP and IEG to go much farther in building and developing evaluation capacity in developing countries," said Guy Laforest, the Executive Director of ENAP. “I am pleased that our school's expertise in evaluation is now available on a wider scale, especially in this year of the 10th anniversary of the PIFED.” “IEG looks forward to strengthening its partnership with ENAP in order to generate synergies and thus extend the impact of the monitoring and evaluation support programs of our two institutions,” said Alison Evans, the Director General of IEG. “This partnership will enable us to support more governments and institutions in strengthening their systems and capacity to support data-based decision-making and results-based information, and thus accelerate their progress towards the sustainable development goals (SDGs).” As a step towards signing the MoU, ENAP hosted an event during the 2020 gLOCAL Evaluation Week, which took place from June 1 to 5, and was organized by IEG and the CLEAR Initiative. ENAP held a joint panel with the Francophone Evaluation Network of Canada and International Program for Development Evaluation Training (IPDET) on the theme "The Online Transition of Pandemic Assessment Capacity Building Practices: Challenges, Opportunities and Limits" in which hundreds of people participated. Note: This is a translated version of the original news story in French. Sign up to receive updates about the growing global partnership to close the gap in monitoring and evaluation capacity worldwide.  

Addressing administrative data gaps in India to fight COVID-19 (coronavirus) and speed recovery

Web Resource
Mumbai, Maharashtra, India,2020. Installing Aarogya Setu app on mobile phone under home quarantine. Launched by government of India for tracking prevention & testing of Coronavirus (Covid-19) pandemic
The authors work with CLEAR South Asia, hosted by Abdul Latif Jameel Poverty Action Lab (J-PAL), South Asia As the world moves to ease restrictions imposed for COVID-19, it is increasingly clear that we will need to learn to live with the virus for some time and make significant behavioral changes. Governments have a difficult role to play in balancing public health concerns with economic needs Show MoreThe authors work with CLEAR South Asia, hosted by Abdul Latif Jameel Poverty Action Lab (J-PAL), South Asia As the world moves to ease restrictions imposed for COVID-19, it is increasingly clear that we will need to learn to live with the virus for some time and make significant behavioral changes. Governments have a difficult role to play in balancing public health concerns with economic needs and are grappling with questions related to how and what to open. Apart from these immediate challenges there is a need to hasten the economic recovery and build greater resilience to fight future pandemics. India has made significant progress in the use of data to guide its strategies for coping with the impacts of the pandemic, a tool that will prove as useful for navigating the reopening. In view of its critical importance, CLEAR South Asia has been working with state governments in India to build capacity to collect and evaluate data better. The COVID-19 pandemic, more than ever, has demonstrated the value of data in addressing public health crises. Researchers in the US analyzed anonymized cell phone location data to understand which enterprises can be ‘super spreaders’. India Observatory has developed a GIS-enabled dashboard to show the movement of migrants in real time and to identify relief centers on their routes. Moreover, governments are using data in different forms for contract tracing and isolation. Good quality data, particularly administrative data, can become a vital tool for governments to plan reopening and rebuilding the economy.              Enlarge and download this infographic Administrative data, or admin data, which is data collected during routine transactions, can be a rich and inexpensive source of information to generate useful research, especially when data from different sources is combined (e.g. mobile signal data and incidence of disease) for easy use. However, in developing countries like India, there are significant challenges with regard to accessibility and usability of admin data, especially when government is the data provider, which limits its use. Access to admin data for research use in India is largely driven by individual champions rather than a comprehensive legal framework or protocols to govern access, storage, transfer and use in a transparent manner. In addition, there are constraints due to capacity and knowledge in making data available to researchers in a secure manner. For instance, ensuring a secure means of data transfer, determining sensitivity level of data fields, meta-data documentation, ability to anonymize data at source are a challenge for data providers, especially governments. Another barrier, particularly for government generated admin data, is its usability in terms of appropriate formats, standardization of collection process and quality. Digitized data in PDF formats or at an aggregated level are not very useful for research purposes. Often, codes are not standardized even for basic geographical units (such as districts, villages) across datasets in India which makes combining datasets difficult. Another challenge is ascertaining data quality in regard to its reliability and accuracy. Given the growing importance of data, the Indian government is increasingly aware of these gaps. A data protection bill is under consideration by the Parliament. The government released data protocols to address privacy concerns under its contract tracing application Arogya Setu. Moreover, NITI Aayog, a premier think tank of the Government of India, has recently launched its vision for the National Data and Analytics Platform (NADP) to address some issues around data usability. While most of these efforts are concentrated at the national level, there is also scope and demand for capacity building at the state level (federal units in India), where implementation takes place. CLEAR South Asia has been working actively with state governments in India to address issues related to access and usability of government data (both primary and admin). We have conducted customized trainings and hands-on workshops for government staff on conducting independent data audits and quality checks. We have also provided advisory support to our government partners on their data policy, for transitioning to digital data collection and improving the data collection/recording process. In one such partnership, the CLEAR/ J-PAL South Asia team provided advisory inputs into the design of a new data collection system that the department was looking to transition to for recording complaints received on their women’s helpline. Our team also helped the government with more standardized formats for recording information in the interim. Going forward, as CLEAR South Asia center, we plan to intensify our engagement with state governments and other data providers to demonstrate the potential of data use and in the process strengthen the access and usability of their admin data systems. Our team will ramp up efforts to provide customized capacity building workshops, and/or advisory services on how to make data more accessible for research, strengthening data collection processes, instituting systematic data quality checks as well as strengthening its use for decision-making to address specific needs and help in course correction. Through these efforts, we hope to support the government’s increasing use of data to inform their decision-making process and as a foundation for rigorous evaluations. We hope that through these sustained efforts of CLEAR SA and our government stakeholders we can build better data systems to help recover and fight the next pandemic.   Pictured above: Mumbai, Maharashtra, India,2020. Installing Aarogya Setu app on mobile phone under home quarantine. Launched by government of India for tracking prevention & testing of Coronavirus (Covid-19) pandemic.  Image credit: Shutterstock/ PhotographerIncognito

Rewiring Evaluation Approaches at the Intersection of Data Science and Evaluation

Web Resource
rewiring evaluation event
These sessions will feature a combination of evaluators and data science users and practitioners to explore lessons, opportunities and challenges in rewiring evaluation. These sessions will feature a combination of evaluators and data science users and practitioners to explore lessons, opportunities and challenges in rewiring evaluation. 

gLOCAL Evaluation Week 2020: A Global Knowledge Movement to Advance Monitoring and Evaluation Knowledge Sharing

Web Resource
With the ten-year countdown to the Sustainable Development Goals (SDGs) underway, it is more important than ever for countries to have the systems and capacities in place to track the progress of their national development strategies, evaluate their impact, and learn from evidence. The new circumstances posed by the Covid-19 pandemic have highlighted even more the crucial role played by Show MoreWith the ten-year countdown to the Sustainable Development Goals (SDGs) underway, it is more important than ever for countries to have the systems and capacities in place to track the progress of their national development strategies, evaluate their impact, and learn from evidence. The new circumstances posed by the Covid-19 pandemic have highlighted even more the crucial role played by monitoring and evaluation (M&E) in designing and developing adequate policy responses, the importance of learning from evidence, and sharing this knowledge across the board. With the objective of fostering a culture of M&E learning and knowledge-sharing and sustaining the momentum of global efforts to promote M&E capacity, the CLEAR Initiative will convene the second annual gLOCAL Evaluation Week from June 1st to 5th.   gLOCAL Evaluation Week 2020 is fully virtual and will feature more than 250 events around the world organized by local public, private, and academic institutions and organizations that produce, use, or promote evaluations to strengthen the impact of development programs. In addition to the traditional thematic areas such as Evaluation Capacity Development, Evaluation Communication and Use, and Evaluation Methods, the special theme, ‘Evaluation 2030’, intends to encourage discussions about the role of M&E in supporting the achievement of long-term development outcomes such as the 2030 Development Agenda and the Sustainable Development Goals. There will be several events exploring M&E themes in various sectors, such as gender, climate change, health, among others.  We also have several events that explore the implications of and for M&E in times of Covid-19. {"preview_thumbnail":"/sites/default/files/Data/styles/video_embed_wysiwyg_preview/public/video_thumbnails/MquEulFC5XE.jpg?itok=6HyjLW3K","video_url":"https://youtu.be/MquEulFC5XE","settings":{"responsive":0,"width":"854","height":"480","autoplay":0},"settings_summary":["Embedded Video (854x480)."]} Sharing local and global M&E knowledge and experiences are essential to strengthen capacity, foster accountability, promote evidence-based decision making, and learn from results. These goals are central to the CLEAR Initiative’s mission and the inspiration for gLOCAL Evaluation Week. -Sophie Sirtaine, Director of Strategy and Operations, Independent Evaluation Group of the World Bank Group, and CEO of the CLEAR Initiative.  Last year, 177 institutions around the world supported the 2019 inaugural gLOCAL Evaluation Week during which over 270 events were organized in 38 countries. An estimated 22,000 participants convened to learn and share M&E knowledge. This year, along with the CLEAR Centers, our partners and collaborators are rising to the challenges posed by Covid-19 with a great deal of innovation and adaptability by making optimal use of technology to bring knowledge and experience sharing opportunities closer to you than ever before. The virtual nature of gLOCAL 2020 will allow organizers to reach wider audiences and enable M&E professionals and organizations to engage in a larger and continuous dialogue across time-zones. This is aligned with CLEAR’s commitment to support the exchange of M&E knowledge and experiences to promote evaluation capacity development, support evidence-based decision making, and strengthen development outcomes at local and global levels. As we get ready for gLOCAL Evaluation Week 2020 next week, this is how you can be involved in gLOCAL 2020: Promote gLOCAL Evaluation Week through your networks and channels. Share news and social media updates about gLOCAL on your social media and other platforms. More information is available at the CLEAR Initiative. Participate in gLOCAL Evaluation Week and encourage everyone to participate. Go to gLOCAL Evaluation Week to see the 2020 Calendar of Events. Share with us and others your experience of gLOCAL 2020. Write to us and follow us on social media. For more information and to see the calendar of events, please visit gLOCAL Evaluation Week 2020 Calendar of events and the CLEAR Initiative, follow the CLEAR Initiative (on twitter @theCLEARInitiative), or email clear@worldbank.org    

A global effort is needed to ensure all countries are ready to combat COVID-19 (coronavirus) with evidence

Web Resource
A global effort is needed to ensure all countries are ready to combat COVID-19 (coronavirus) with evidence
Every government needs robust monitoring and evaluation (M&E) systems now more than ever to design effective policies.Every government needs robust monitoring and evaluation (M&E) systems now more than ever to design effective policies.

Of Mice and Men (and World Bank Projects): Harnessing Behavioral Approaches

Web Resource
Soap on a rope. A recent experiment in Zambia explored how giving students soap-on-a-rope to take as an entry pass to the bathroom led to significant increases in handwashing behavior among the children.
Behavioral Science offers insights and design principles that can be incorporated into existing programs, often at low cost, to increase program reach, effectiveness, and sustainability.Behavioral Science offers insights and design principles that can be incorporated into existing programs, often at low cost, to increase program reach, effectiveness, and sustainability.