Better Together? Seven Guiding Principles for Evaluating Partnership Programs
Ensuring everyone involved in a partnership evaluation gets what they expect
Ensuring everyone involved in a partnership evaluation gets what they expect
By: Caroline HeiderWorking in partnership is today more common and more important than ever before, and given the complex challenges the development community is dealing with the need for effective partnerships is only going to increase.
For IEG, a longstanding partnership forum has been the Network on Development Evaluation which regularly brings together bilateral development partners. During the network’s most recent meeting last month, I spoke about what IEG has learned about evaluating partnership programs based on a large number of evaluations of programs in which the World Bank Group is involved.
1. To improve the authorizing environment, make sure there is a mutually agreed evaluation policy.
Many partners will have different ideas about evaluation. Agreeing, right from the start, what will be evaluated, by whom, when and why is important to ensure evaluations are conducted in a timely manner. If it is a self-evaluation, then arrangements for an independent validation or review are needed to ensure credibility.
2. To improve credibility, ensure evaluation independence.
How much independence is necessary? The answer to that question will vary considerably across multilateral development institutions and bilateral donors. Varying standards of independence among partners is often a source of confusion and has a direct impact on evaluations of partnership programs. That’s why it is important that partners develop a shared understanding about what are the acceptable standards for independent evaluation.
3. Invest time in planning the evaluation.
Partnership programs often cover a wide range of activities, all of which need to be assessed by the evaluation. Frequently, the partnership arrangements themselves, from governance, to funding and staffing issues, will also be evaluated. Defining the scope and coverage of the evaluation in clearly defined ways is necessary to ensure that partners share expectations, that their most important questions are answered, and that adequate resources are allocated for the exercise. The clearer the boundaries, the more focused the evaluation will be.
4. Choose criteria that fit the purpose.
Evaluations of partnership programs use two broad sets of criteria: one for the development outcomes of the partnership program itself, and the other for its organizational effectiveness to measure how well the partnership is functioning. In assessing outcomes, evaluations build on the standard evaluation criteria established by the OECD/DAC in 1991. For questions of organizational effectiveness, the focus is on governance structures - sharing of voice and power on one hand and of financial burden-sharing on the other - and on whether the partners are in fact getting what they expected out of the partnership.
5. Make sure the evaluation is transparent and that key stakeholders are consulted.
It’s not enough for an evaluation to be properly carried out – if it is to have an impact, its findings need to be well received, discussed, and acted upon. Transparency is paramount in this context. If any of the partners or stakeholders do not understand the evaluation process or the criteria against which activities are evaluated, they may reject the findings, however sensible they may be. Consultations with stakeholders – from all different perspectives and interest groups – are essential. They will also help to ensure that the findings don’t come as a surprise.
6. Ensure that recommendations are agreed on and followed-up.
As with all evaluations, recommendations should derive from findings and conclusions based on evidence. In partnership programs, these recommendations might be directed more towards one group than another. In general, the wider the scope of these evaluations, the harder it can be to limit the recommendations to those that are the highest priority and where their implementation can be monitored.
7. Plan dissemination in advance.
Making a difference with an evaluation can also require thinking beyond the main stakeholders. Broader sharing of the evaluation’s findings through the internet and various other means is an important aid to learning. Unfortunately, we’ve found that this is something that too few of these evaluations do.
These principles are just a starting point in evaluating the effectiveness of partnership programs. Ensuring that both partners in these programs are in fact “better together” is something that we at IEG together with EvalNet will continue to work on.
Comments
Add new comment