Search

Content type
Displaying 71 - 80 of 5428

Placing Gender at the Heart of Evaluation

Web Resource
placing gender at the heart of evaluation, inclusive agenda, assistance to low-income countries, conflict affected states
The gender impact of a development intervention may not always be easy to see, but it’s rarely absent. The gender impact of a development intervention may not always be easy to see, but it’s rarely absent.

Selfies in Evaluation: Improving the Project Self-evaluation System

Web Resource
How to make self-evaluation more candid and less burdensome How to make self-evaluation more candid and less burdensome

Who Evaluates the Evaluators?

Web Resource
Though more used to telling others how they’re doing, evaluators must also be willing to be assessed if they’re to be credible.Though more used to telling others how they’re doing, evaluators must also be willing to be assessed if they’re to be credible.

Breaking News: Evaluation on Prime Time TV

Web Resource
Four evaluators walk into a TV studio. What happens next says a lot about the field’s growing influence. Four evaluators walk into a TV studio. What happens next says a lot about the field’s growing influence.

Keeping the Focus on Gender Equality

Web Resource
gender equality, focus on gender
Ahead of International Women’s Day, I argue that in order to realize the Bank’s twin goals we need to internalize gender equality into our solutions and our evaluations.Ahead of International Women’s Day, I argue that in order to realize the Bank’s twin goals we need to internalize gender equality into our solutions and our evaluations.

When Rating Performance, Start with Yourself

Web Resource
Faced with growing demands for evidence-based decision making and measurable results, do we need to look for new ways to evaluate the evaluators? Faced with growing demands for evidence-based decision making and measurable results, do we need to look for new ways to evaluate the evaluators?

IEG as a Learning Organization

Web Resource
How do you ensure that lessons learned from evaluations are reaching those who could most benefit? An initiative begun by IEG staff in 2013 is already showing results. How do you ensure that lessons learned from evaluations are reaching those who could most benefit? An initiative begun by IEG staff in 2013 is already showing results.

Measurement Mantra

Web Resource
The World Bank Group has a proud history of empirically-based decision-making and advice to client countries, leadership in results measurement and aid effectiveness, ground-breaking work in impact evaluation and Open Data, and the longest standing independent evaluation office in international development. Against this backdrop, it is surprising that IEG finds that results frameworks and Show MoreThe World Bank Group has a proud history of empirically-based decision-making and advice to client countries, leadership in results measurement and aid effectiveness, ground-breaking work in impact evaluation and Open Data, and the longest standing independent evaluation office in international development. Against this backdrop, it is surprising that IEG finds that results frameworks and monitoring and evaluation in WBG operations are inadequate more often than not. Are the results frameworks to blame or is something else causing the problem? And, how will the 2013 World Bank Group Strategy succeed in embracing evidence, measurement, real-time learning, and evaluation to ensure the transition to a ”solutions bank?” Here are some highlights from recent IEG evaluations about lessons learned – or not – from self-evaluation systems. At the Intervention and Project Level. The World Bank Group is the largest producer of impact evaluations (IEs) among all development institutions – by the end of 2011, 460 were either completed or under way.  Our 2012 evaluation of IEs highlighted positive trends: increasingly, IEs are incorporated into project design and used to test variations in project components and how they produce different outcomes, or to assess distributional and welfare impacts. However, the evaluation also showed that only between 22 and 33 percent of IEs were used in operational decisions, and only around 50 percent had an influence on policy dialogue. Why?  To get your IE used, here are some important factors that play a role: ask the right questions, actively involve local stakeholders, apply good practice quality standards, timeliness, and appropriate dissemination.  The quality dimension is particularly important -- our 2013 Systematic Review of Mother and Child Mortality highlighted that only 68 of some 7,000 studies met international standards, among them 18 in which the World Bank had some involvement.  A culture of evidence-based decision-making and a functioning system to link evaluation results to decision-making processes help as well. At the Country Level. Any of you who have read country program evaluations and reviews of country strategy completion reports know that producing high quality results frameworks in country strategies is difficult.  Together with indicators and measurement systems, they often result in downgrading ratings for performance and results.  In a number of cases, country strategy progress reports did not result in changes, even when they flagged that all expected risks had materialized.  To address any problems that might result from using different yardsticks, IEG agreed with Bank management on a common rating system. In addition, we are deepening our analyses to explain why results frameworks are weak and how these shortcomings can be overcome in the new Country Partnership Frameworks, and are working on a prototype learning product that will integrate lessons from independent evaluations into country strategy formulation. At the System Level. The 2013 evaluation of the International Finance Corporation’s (IFC) monitoring and evaluation systems traced poor results to poor quality at entry. In particular, it observed that in 60 percent of unsuccessful IFC investment projects, appropriate and relevant lessons had been identified at the early review stage, but had not been integrated into project design.  IFC advisory services, likewise, suffer design shortcomings such as a lack of proper indicators and baseline data, unclear objectives, and unrealistic expectations in outputs and outcomes.  This occurs despite a wealth of lessons from project completion reports, external impact evaluations, and IFC’s SmartLessons.  To incentivize behaviors that focus on development outcomes and learning, in 2004 IFC developed a long-term performance awards program, and incorporated IFC’s Development Goals into the corporate scorecard.  The evaluation also flagged tensions between corporate indicators and those that make sense in the context of a country or project. At the Corporate Level. The 2012 Results and Performance (RAP) report commented on the Bank’s corporate scorecard, recognizing it as a significant improvement in capturing corporate results. The RAP nonetheless flagged a number of areas for attention, including capturing the results of knowledge work. It also suggested that indicators needed to be expanded to capture the Bank’s results at the country level and that the system needed to allow disaggregation to generate meaningful information and lessons. The WBG’s change process involves developing one integrated score card and results framework – an exciting opportunity to overcome these challenges and demonstrate global leadership in measuring and reporting development results.

External Evaluation – A Guarantee for Independence?

Web Resource
Why is independence so important? The simple answer is trust. Public institutions, such as the World Bank Group, need to account for the resources entrusted to them and the results they promised to achieve. Self-reporting and Open Data are essential to increasing trust. So is independent evaluation which validates the truthfulness of self-reporting and delivers greater in-depth analyses to Show MoreWhy is independence so important? The simple answer is trust. Public institutions, such as the World Bank Group, need to account for the resources entrusted to them and the results they promised to achieve. Self-reporting and Open Data are essential to increasing trust. So is independent evaluation which validates the truthfulness of self-reporting and delivers greater in-depth analyses to understand the reasons for achievements and shortfalls. As one of my predecessors, Yves Rovani, put it in 1988: "Internally, our independence helps ensure the choice of subjects, access to information, and candor of reporting necessary for accessing accountability and for drawing fully on the benefits of experience. Externally, it contributes to the credibility of the Bank's achievements by demonstrating publicly its openness and willingness to be the first to recognize its own shortcomings." Contrary to providing a second opinion, independent evaluation assesses, as objectively as humanly possible, the success and failure of policies and interventions, and reports critical findings without fear of repercussion. Independence, for that matter, serves to safeguard this objectivity of evaluation. How best then to limit subjectivity and influence over evaluation? It sounds straight-forward: get people who are external to the institution to evaluate it, as implied in the Washington Post article U.S. push for outside oversight over the World Bank. The "internal – external" question can be very basic: you are "internal" when you work for the institution whose policies and interventions are evaluated, or you are not an employee and you are "external".  Underlying the idea that "external" is more independent are assumptions that being outside the line of command and pressures within the institution, separate from corporate culture and group think, enhances independence and impartiality, because interests might be less vested. Independence has been a constant in my career as evaluator. I have worked for many different institutions, each of which managed the quest for impartiality in different ways, but the fundamental principles were the same: reduce biases to the greatest extent possible and report on success and failure, even when "bad news" is not welcome. Recognizing the reality of these pressures and the need to manage them as well as perceptions of independence, extensive discussions about evaluation were held within the World Bank Group in 1973 – many of which were led by the U.S. government.  These discussions resulted in the establishment of an evaluation function outside of the institutional structure responsible for operational matters. Two years later, in 1975, independence was further strengthened. The Director General, Evaluation, reports directly to the Bank’s shareholders – the Executive Board – and has limits on the terms of service with no right to further employment at the World Bank. Other aspects that can limit independence – the choice of what is evaluated, the budget, and freedom to publish reports without censorship – were all addressed with similar measures. These principles became accepted as "gold standards" in the evaluation community and were later adopted by regional development banks and other international finance institutions. Is this "external" enough? Admittedly, in all of my positions, I have been an "internal" evaluator – in that I was employed by the institution - and worked with "external" evaluators in varying roles. Yes, the external evaluators bring a different perspective and are further removed from implicit norms. They might be more impartial, although there is no guarantee of that. Most importantly, it takes an outsider a lot longer to understand institutions, find information, establish trust, and become effective influencers. IEG’s long-term agenda allows us to have the greatest impact on the World Bank’s development effectiveness by making strategic choices about what is evaluated, conducting evaluations that are innovative and cutting edge and that influence stakeholders during the evaluation and not just once the report is issued. We are then able to share lessons gained from experience to foster learning in the World Bank Group and with its partners.  Our long-standing history of independence and the institutional measures in place to safeguard it have both been recognized as instrumental to increasing the credibility of the World Bank Group, which is more than an external evaluation could do.

What is the Value of Independent Evaluation?

Web Resource
Organizations like drivers can suffer from blind spots. Independent evaluation can help provide a broader view. Organizations like drivers can suffer from blind spots. Independent evaluation can help provide a broader view.