Evaluators must make strategic choices at every stage of the evaluation cycle. This matters because value can be generated or lost - and costs incurred or saved - throughout the lifecycle of an evaluation.   While some of the factors that affect value creation are outside the control of evaluation, there are plenty of things that can be anticipated and managed by evaluators.

There are three distinct questions that evaluators can ask in order to create value rather than destroy it.

• What to evaluate and when;
• How to evaluate; and
• With whom and how to share evaluation results, insights, and knowledge.

In this part of our value for money (VfM) of evaluation series, I want to unpack each of these questions, starting today with the importance of making strategic choices on what to evaluate and when.

What to Evaluate When.  The choice of what gets evaluated is particularly important for VfM: value and cost (money) are driven by what gets evaluated and how. This is even more so when looking at the portfolio of evaluations of an institution. Choices need to balance two sets of considerations.

• Coverage of the institution’s work to generate evidence and evaluation of the health of the institution as a whole; with
• Strategic issues, which once deeper evidence is generated, can be the biggest change agents for institutional results and performance.

This balance can be achieved by the right combination of business lines that focus on different units of accounts (from projects through country strategies to policies and corporate issues) with different types of evaluations (validation of self-evaluation, independent evaluation, and synthesis products that build on existing evaluation evidence).

Mirroring the Institutional Portfolio. To evaluate the health of the portfolio overall, one needs adequate evaluation coverage of an institution'€™s work. To achieve this, it is important to understand its entire portfolio of activities. More often than not, it will take several variables in combination to describe the portfolio. They can include size, geographical location, sector, type of instrument. In other words: characteristics that need to be taken into account to ensure representativeness of the overall portfolio. For instance, if an institution works predominantly in Africa, but the evaluation portfolio is focused on Asia, the results would not be an accurate representation of the health of the institution's portfolio as a whole. The challenge lies in finding the right balance between evaluating enough to ensure appropriate coverage, but not so much that costs become disproportionate. In some cases, a 100% sample makes no sense, in others it is needed to provide in-depth analyses of important aspects of the portfolio.

Making Strategic Choices. For other evaluations, it is important to understand which of them can make a strategic difference.  This is particularly so for large, costly evaluations. Their value derives from (and their cost can be justified) their ability to bring about evaluative evidence and insights that can stimulate strategic or systemic changes. These evaluations might tackle controversial issues or focus on aspects of ongoing change processes where just-in-time evaluation evidence can help inform decision-makers' choices. But, there is no simple way when to do one or the other, and not just one thing that gets you to the answers about what's strategic.

Ask Your Stakeholders. Stakeholders who want to use evaluation evidence need to receive that information at the time when they are debating and deciding what to do. It is therefore important to understand

• The issues they are tackling. In some cases, stakeholder readiness to deal with an issues is crucial to materialize the value that evaluation evidence brings to the table. But only, if they need evaluation evidence to understand the direction of change. In other cases, it might be more important to evaluate controversial issues that (some) stakeholders would rather not see addressed; and
• Decision-making processes, especially milestones when evaluation evidence is needed. It takes thinking ahead – as much time as it takes to complete an evaluation -€“ to pick up strategic issues in a timely way and have an evaluation available right in time for decision-makers' use.

Understand the Past with a View to the Future. When an institution is making a strategic shift, an evaluation of past policies and programs will help understand how far the institution has come, how big and where the gaps are. There are sensitivities that need to be managed well when making strategic choices of this nature. For one, the vision for the future has to be settled, to ensure the evaluation applies the right lens for the future. Otherwise, it will not generate its potential value, but waste resources. In addition, it is important to manage well the concern that the past will be evaluated against a new yardstick. In such case, the risk of damaging relationships is high -€“ a cost that might outlive the specific evaluation - and might outdo the value of evaluation evidence that helps prioritize areas for improvement.

Tackling Complex Issues with a Set of Evaluations. Some issues are fundamental to moving an institution to the next level. But they are too big and complex to tackle at once. Instead, a structured program of strategically selected evaluations can help shine light on the issue from different directions. This is exactly what we are trying to achieve with the Strategic Engagement Areas, which were selected as fundamental for achieving the Bank Group'€™s twin goals, but too big for a sole evaluation.