After attending the UNDP National Evaluation Capacities and the American Evaluation Association Conferences in 2019, I wrote a blog Are Evaluators Ready to Answer Question: Who Benefits? The answer was that increasingly evaluation practice was grappling with ‘Who Benefits’ through increasingly engaging with issues such as inequality, inequity, the differing impacts of climate change, and the role of youth. This year I attended the virtual American Evaluation Association (AEA) conference interested in how the debates on assessing benefits had developed.

The theme of the AEA conference was ‘How Will You Shine Your Light’, which relates to the idea that evaluators shine their light for the purpose of “improving conditions for others to shine theirs”. Throughout, the conference focused on equity, the fair distribution of benefits. The presidential track of the conference reflected on equity through discussions on restorative practice, racism in qualitative practice, disability in evaluation, community-based foresight practices, climate change and presenting a framework for assessing community harmony. Outside of the presidential strand, presentations also focused on gender equity, equity-based policy advocacy and equity and inclusion.

The emergence of equity as a major theme in evaluation practice reflects trends where evaluation gatherings embrace a diversity of perspectives, including indigenous, youth and social justice perspectives. Many of the presentations at the conference reflected critically on approaches to engage marginalized groups and called for more context specific and culturally appropriate methods. Some of the interesting resources that I would reference again as I develop evaluations were related to youth, the rubric developed to measure love, equity in data science and foresight techniques.

My understanding of the ‘Who Benefits’ question was especially deepened in facilitating a Think Tank entitled Doing No Harm in Evaluation that drew on a two-year action learning process. Discussing harm is the ‘shadow’ of the equity and benefits discussions, as one of our discussants highlighted. Evaluators and project managers who have the best of intentions and seek to diversify who, what and how benefits accrue can do harm – think of abuse in humanitarian settings.

In debating harm, the participants went beyond concerns with ethics panels and informed consent. They questioned how harm could be caused by funding and commissioning mechanisms, a tendency to drop into contexts with limited preparation, and the answering of questions that were defined far from intended beneficiaries. Emerging research that was presented within the Think Tank made the argument that in evaluation there is inherent reductionism and carcerality, which requires changing listening practices, reciprocity with knowledge, and undertaking truth-telling on harm.

The online chat during the think tank produced just less than 300 comments in 45 minutes. In Interpreting these afterward the following five themes emerged:

• Trade-Offs – In identifying who benefits, harm is embedded. Consequently, trade-offs between benefit and harm should be acknowledged in the evaluation processes, for example, in the (i) methods deployed; (ii) scope of the evaluation; (iii) engagement with commissioners; (iv)  management process.
• Reflexivity – Evaluators hold a position of privilege in defining benefit and should give attention and attend to personal growth in their own judgements, beliefs and practices as benefits can be defined in diverse ways.
• Trauma informed and restorative practices – Identifying who benefits may also bring an evaluation into contact with past traumas. Trauma-informed and restorative practices arose as approaches to help account and tell the truth on the harms of past practice while providing a framework to help manage trade-offs in current evaluation practice.
• Inclusivity – To understand who benefits requires improvements in h who evaluation practice includes, through listening skills and incorporating feedback loops. Creating space for listening will require responding to new and unexpected information on diverse perspectives.
• Transformative lens – Working with a transformative evaluation perspective helps to identify different benefits and harms by reinforcing attention to voice and liberation in discussions. Transformation provides potentially a useful ethical principle in reframing practice to better respond to harm.

These themes and the discussions presented here reflect a small part of exchanges on equity that evaluators engaged in during the conference. Terms such as trade-offs, reciprocity, restorative, reflexivity, transformative are increasingly eminent in evaluation practice and represent a deepening discussion amongst evaluators to engage with ‘Who Benefits?’

Pictured above: High peak demand for energy in FYR Macedonia can result in power outages around the country. Here a young child does homework by lamplight in a home outside of Skopje. Photo: Tomislav Georgiev / World Bank