Box 1.1. Key Terms and Concepts

Outcome: A change in behaviors, conditions, or situations resulting from World Bank Group activities. Outcomes may include intended, unintended, positive, and negative changes.

Theory of change: The logic, expressed in project design documents and the Implementation Completion and Results Report, that identifies expected cause-and-effect relationships among inputs, activities, outputs, intermediate results, outcomes, impacts, and underlying critical assumptions.

Project development objective (PDO): A World Bank project’s stated objective, framed as a positive outcome.

Project claims and market and foreign investment claims: The Independent Evaluation Group considers these equivalent to the World Bank’s PDO in the Anticipated Impact Measurement and Monitoring system of the International Finance Corporation (IFC) and the Impact Measurement and Project Assessment Comparison Tool system of the Multilateral Investment Guarantee Agency (MIGA).

Self-evaluation: A formal assessment of a project, program, or policy conducted by or for those in charge of the activity. In the Bank Group, self-evaluation takes the form of a systematic written account of the results and performance of a project or operation, and those in charge assign ratings based on criteria defined in guidelines to assure comparability among reports.

Validation: The Independent Evaluation Group’s independent, critical review of the evidence, results, assessments, and ratings from self-evaluation.

Ratings: Ratings are rubrics for assessing performance relative to a project or program’s objectives. Ratings summarize the self-evaluation narrative into categories or values that enable aggregation across operations. Examples include the following:

• For World Bank projects, the “outcome” rating brings together three underlying dimensions: relevance, efficacy (achievement of objectives), and efficiency. Independent Evaluation Group validations assign ratings for a project’s efficacy in achieving each of its individual objectives and for overall efficacy in achieving the project development objective. Other key ratings are quality at entry (which, together with the quality of supervision rating, determines the Bank performance rating) and monitoring and evaluation quality.
• For IFC investment projects, the “development outcome” rating brings together four underlying dimensions: project business success, economic sustainability, environmental and social effects, and private sector development. IFC’s development outcome ratings do focus on the achievement of expected objectives and a project or company’s results against several benchmarks (such as the performance of peers, the market, or similar industries) and unintended outcomes (positive and negative). Other key ratings are for IFC’s additionality, IFC’s investment outcomes, and IFC’s work quality.
• For IFC advisory services projects, the “development effectiveness” rating brings together five underlying dimensions: strategic relevance, outputs, outcomes, impacts, and efficiency. Other key ratings are IFC’s work quality and IFC’s role and contribution.
• For MIGA projects, the “development outcome” rating brings together four underlying dimensions: project business success, economic sustainability, environmental and social effects, and foreign investment effects. Other key ratings are MIGA’s strategic relevance, MIGA’s role and contribution, and MIGA’s assessment, underwriting, and monitoring, which are aggregated under MIGA’s effectiveness ratings.

Monitoring and evaluation (M&E) quality: For World Bank projects, M&E quality is assessed at the project level and comprises M&E design, implementation, and use.

• M&E design is assessed based on the extent to which (i) the theory of change was sound and reflected in the results framework; (ii) the objectives were clearly specified; (iii) the indicators encompassed all outcomes of the PDO statement; (iv) the intermediate results indicators could adequately capture the contribution of the operation’s components (activities) and outputs toward achieving PDO-level outcomes; (v) the indicators were specific, measurable, achievable, relevant, time-bound, and had baselines and targets available; (vi) the measurement methods were adequate; and (vii) the arrangements were well embedded institutionally.
• M&E implementation is assessed based on the extent to which (i) planned baseline data collection was carried out; (ii) the indicators included in the results framework were measured and reported; (iii) weaknesses (if any) in M&E design—including specification of indicators—were corrected during implementation; (iv) the agency responsible for M&E (and any other relevant stakeholders) ensured attention to effective M&E implementation; (v) data used for M&E were found to be reliable and of good quality (important elements here include sound methodology, independence of analysts, and quality control); (vi) if relevant, beneficiaries were involved in defining target indicators and assessing their achievement; and (vii) M&E functions and processes are likely to be sustained after project closing.
• M&E use is assessed based on the extent to which (i) M&E findings were communicated to the various stakeholders (for example, to inform adaptive management); (ii) M&E information led to strategic redirection or resource allocation or to other positive or negative shifts in the implementation of the project or program; (iii) M&E data were used to provide evidence of achievement of outcomes and not just to provide evidence of application of inputs or achievement of outputs; and (iv) M&E data or findings have informed subsequent interventions or are expected to influence subsequent interventions in the near term.
• High M&E quality helps clarify the “line of sight” from projects to high-level country outcomes by explicitly defining and demonstrating project-level outcomes.

Source: Independent Evaluation Group.