How do we ensure that evaluations include gender and equity?

Let’s not forget the role that people play! It is people who use evaluation as a tool to illuminate and address inequities: guidelines, systems, and capacities are insufficient. 

Let me give an example from my work.

We talk a lot about disaggregated data.  But just having that data does not mean it will be used to highlight inequities.  In Bihar, India, we are working with the government and other partners on an ambitious maternal and child health program.  The program captured baseline data in early 2012 from women who gave birth the previous year across the State. Over 13,000 women were surveyed. We had data on gender, birth parity, caste, income, religion, etc.  But that does not mean we had gender and equity covered.  Someone has to do and use the analysis. That takes resources and is a choice.

When I joined the foundation the baseline report had been written. It had some sub-group analysis but gender and equity had not been a core focus.

The program team wanted to reduce inequities but data was not being fully used to deepen this interest.

We ran additional analysis.  The following chart is one example of what we found.

hay_blog_chart1_1.gif

 

Although we knew that most women (72%) on average were not getting adequate antenatal care (ANC), this analysis showed large variations among women.  Only 14% of illiterate women from scheduled caste, schedule tribe, or Muslim communities in the lowest economic bracket received three antenatal check-ups compared to 57% of literate women from less marginalized, and higher income brackets.

At this point the midline was upon us.  We were only weeks away from being able to examine whether inequities had been reduced two years on.

With more detailed analysis of inequities in hand, we asked the team whether they expected the program to reduce inequities, hold them constant, or widen them. This created a surge of interest in results for marginalized groups at midline.  Evaluation can lead as well as follow.

In hindsight the choice to deepen this analysis and use it to engage with programs sounds like just doing good evaluation.  And it is.  But the point is that it doesn’t just happen.  We make choices about what to look at all the time.

How did that story end?

Wonderfully. At midline, they had substantially improved outcomes for marginalized women in several areas including front line worker visits, breastfeeding, complementary feeding, and use of modern methods of family planning.  That’s not easy to do.

Obviously wide gaps still remain.  But the program teams have a renewed focus, enabled by measurement, on continuing to narrow those gaps.

Another thing we did in that midline was to question on intimate partner violence.  The foundation was not working on intimate partner violence, so why do that?

Evidence cannot take us in new directions if we limit evidence collection to what we are already doing.

Anyone who has fielded a survey knows you can’t include everything.  My usual default is to include the most actionable questions.  By that measure, violence may not have been included.  But evaluation can’t challenge inequities if we don’t make space for them.

So I put in the questions.  Because I could.  And that’s the secret.

We found very high levels of violence (see table).  Almost 50% of the women surveyed reported physical abuse and 17% forced intercourse.  Among the youngest and most marginalized women, 76% reported physical abuse and 33% forced sexual intercourse. We looked at the relationship of this violence to health outcomes. Among several associations, we saw that women facing severe physical violence, though more likely to receive home visits from health workers were less likely to deliver at facilities, and more likely to give birth to low birth weight babies.[i]

hay_blog_chart2_0.gif

I did not wait for someone to instruct me to ask those questions.  That’s not how evaluation works.  I included them because I believed if I had evidence on the factors and inequities underpinning health, my colleagues and partners working to improve women’s health would act.

I was right.

The data came out at a great time. The foundation was deepening focus on gender and equity and developing a Grand Challenge on Putting Women and Girls at the Center of Development.  I could not have predicted this when I inserted these questions.  It doesn’t always work this way.  Sometimes people are not interested; that’s one reason inequities are so persistent.

Melinda Gates recently wrote in Science Magazine, that the Bill & Melinda Gates Foundation will be more intentional about addressing gender inequalities. I share my story to make the point that evaluation also has to become more intentional.

So don’t wait for a guideline to integrate gender and equity into your evaluations. Do it in stealth if you have to. That is how change happens.  My call to evaluators is to be accountable for equity. We are not observers; we are participants in the way development unfolds.   Take a position on equity and be active in measuring it with the tools of our trade.   If we do not bring a gender and equity focus to evaluations, that is on us.


[i] Data will be published shortly. For more information please contact the author.

Comments

Permalink

Dear Katherine, What an excellent blog on a theme close to my heart: Leading Through Evaluation! I just facilitated a workshop with that title at the American Evaluation Conference in Denver using Ron Heifetz Adaptive Leadership model, and merging it with evaluation. For Heifetz, there are no leaders, only acts of leadership. Acts of leadership are those acts that enable social groups and organizations to "get the work done," and you have done just that. We need to get the work done that reveals gender disconnects, gender-based violence, and other inequities. We cannot wait for someone to instruct us (as you wrote), but take the initiative to ask the questions that need asking, and pay attention to the analysis and implications of the data we collect for social justice. These are the courageous interventions that will enable our societies and organizations to face difficult truths and look for ways to do something about them. I would appreciate learning about other examples of such interventions that help our society make constructive steps toward social justice goals. To quote Heifetz, Katherine, "your inspiration taps hidden reserves of promise that sustain people through times that induce despair." Thank you for the inspiration, Katherine.
Permalink

Hello Katherine: Thanks for the powerful, inspiring blog. I like the way you challenge evaluators to get out of their comfort zone and do what needs to be done without waiting for policies and guidelines. It is a nice example of evaluator advocacy that makes perfect sense to me. All the best.
Permalink

What an inspirational article, Katherine! It coincides with today’s AEA-365 blog of Liz Zadnik on Bringing passion and enthusiasm to program evaluation (http://aea365.org/blog/liz-zadnik-on-bringing-passion-and-enthusiasm-to-program-evaluation/?utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+aea365+%28AEA365%29). She also invites evaluation professionals to go beyond standard roles or molds learned in our technical careers,to contribute to bring about social change and community wellness. You are an example of such attitude. These doses of activism are required to break standard rules and have evaluation become more intentional as you propose. On the other side, from the case presented by you it is demonstrated that intimate violence is a health problem and I agree that any evaluation of these programs should include collection of evidence on the factors and inequities underpinning health. Thank you for sharing the experience and I look forward to the publication of this study. Fabiola Amariles - Learning for Impact
Permalink

Hi Katherine Hay, Very informative data sets related to Bihar State of India. The KOSI River Basin of Bihar was the last reservoir of Wild Polio Virus Type 1 that led to extensive campaigns and finally they made it and the place continues to be polio free since 2011. I would like to share my paper - the last reservoir of polio in India. Please let me know if you want me to do this. Regards
Permalink

Dear Tessie, Rakesh, and Fabiola - Thank you so much for your lovely response to my blog and your words of solidarity. More and more evaluation professionals are stepping forward to demonstrate that rigor does not come from distance or disinterest but rather, that deep engagement and commitment can connect with technical rigor in new ways to transform and guide change. It's wonderful to be part of this growing community of evaluators who are working in our own small ways to be part of this transformation. I think it is this work that will draw a new crop of earnest, engaged, and technically excellent young evaluators to the field.
Permalink

This magazine is very helpful because I coordinate World Bank financed projects in Kenya

In reply to by James Theuri

Permalink

James, many thanks. We are always happy to hear that the blog is useful!
Permalink

Hi Katherine, Excellent post, and gruesome statistics on physical abuse of women.I whole heartedly agree with you on the need to ensure integration of Violence Against Women - by stealth if not openly - into our evaluation methods. Just curious about that last bar on "Husband drinks" (37.9%). From my experience in Oxfam, VAW activists have had strong opinions about linking alcohol abuse to VAW, claiming that VAW takes place even when "Husband does not drink" (62.1%). Any learnings about correlations of alcohol to VAW from your survey on this issue? Pankaj

Add new comment

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.