CIPS_blog_logo

Tangible Evidence or Wishful Thinking?

Tangible Evidence or Wishful Thinking?
American delegates to the April 1915 Women’s Peace Conference at the Hague. The delegates include feminist and peace activist Emmeline Pethick-Lawrence (1867–1954), social activist and writer Jane Addams (1860–1935), and Annie E. Malloy, president of the Boston Telephone Operators Union. To the right of Malloy may be labour journalist and activist Mary Heaton Vorst (1874–1966) and the woman wearing a hat on the far right may be Lillian Kohlhamer of Chicago.Bain News Service

“Women’s participation in the peacebuilding process increases by 35% the probability that a peace agreement will last for at least 15 years.” Or does it?

“If you want to make peace sustainable, include women in the peacebuilding process.” Over the last few months, I have seen this claim, or variations of it, everywhere. Canada’s Feminist International Assistance Policy makes it. “Canada’s National Action Plan on Women, Peace and Security 2017–2022” makes it. It can be found on the Global Affairs Canada website in the section on “Women, Peace and Security.” And on the website of UN Women. A recent promotional video of ICAN (International Civil Society Action Network) makes it (and even bumps up the percentage from 35% to 54%).

It is no surprise that this claim resonates so well in a country that brands its foreign policy, its international assistance policy, and even its defence policy as feminist. And it must be tempting for many policymakers to take this claim at face value. After all, it promises a seemingly easy policy solution for oftentimes insurmountable problems, all in line with the ideological tenets (and the branding strategy) of this government.

But what is the evidence for this claim? Let me briefly pause here to explain why I think evidence in public policy is important. Evidence is — or should be — a crucial ingredient for policymaking. We need evidence of what works and what doesn’t. Otherwise, ineffective (or even counterproductive) programs will be funded at the expense of programs that might have been effective. There are always opportunity costs in public policy. Policies based on the wrong evidence are bad policies, even if inspired by good values. Doing the right thing must always include doing the effective thing.

Let’s return to the claim that “women’s participation in the peacebuilding process increases by 35% the probability that a peace agreement will last for at least 15 years.” I first came across this claim when I studied, together with my graduate students at the School of Public and International Affairs, the evidence base for Canada’s Feminist International Assistance Policy.

Here is what we found. The claim is made prominently in Canada’s Feminist International Assistance Policy. A footnote provides a reference to an online publication by UN Women, which indeed makes this claim. The UN Women publication also gives no evidence for the claim, but provides another reference that leads to a 2015 policy report by the International Peace Institute entitled “Reimagining Peacemaking: Women’s Roles in Peace Processes.”

The claim — made in Annex 2, which consists of one page — is said to be based on an analysis of “a new dataset that measures whether or not one or more women participated in peace talks occurring between 1989 and 2011.” The author of the annex, Laurel Stone, does not explain how she reached her conclusion. There is no discussion of data and methods, and no further evidence is given. A Google search reveals that Laurel Stone authored a working paper entitled “Women Transforming Conflict: A Quantitative Analysis of Female Peacemaking” in 2013. The paper appears to be part of her MA thesis at Seton Hall University. The working paper is not peer-reviewed, and I have not found any subsequent peer-reviewed publications by the author.

The paper is based on a statistical analysis, using several statistical regression models, that investigates the effect of women’s inclusion in peacebuilding on the duration of peace. In one model specification we find that “women’s participation” is significantly associated with the observation that the peace holds for one year. However, there is no significant association with peace after five years. And as soon as other factors that may also explain the duration of peace are accounted for — such as the level of democracy in the country, or whether or not a peacekeeping mission was present — the variable “women’s participation” loses its statistical significance in all models. In sum: this paper, which appears to be the original source for a widely repeated causal claim, does not provide any solid evidence that including women will make peace more sustainable.

So here we are. It turns out that one of the core claims of Canada’s foreign policy is not backed up by any evidence. Actually, the original source of the claim (an unpublished MA thesis) even suggests that there is no association at all between the two.

Should we care? I think we should, for three reasons.

First, I believe that my profession — being a social scientist at a policy school — is essentially about analyzing and providing evidence that is as balanced, fair, and rigorous as we can while being very transparent about the limitations of our insights. Sometimes this also includes pointing out that there is simply no evidence for a policy. This is not always fun — it would be much more thrilling to indulge in grand visions and morally superior positions — but we cannot expect someone else to do the “evidence work” for us.

Second, we should care because public policies based on evidence are probably more effective than those based on values (or branding) alone.

Finally, how else are we to learn about how we can best achieve sustainable peace, or any other public policy objective, other than by paying close attention to the evidence? If there is no evidence, we must be transparent about the gaps in our evidence base.

By the way, the Government of Canada appears to be on the same page. The Feminist International Assistance Policy proudly announces that “Canada will step up its commitment to evidence-based decision making […] and will strengthen the evidence-base by investing in policy research, better data collection, and evaluation for gender equality.”

These are good intentions, but so far, we have seen little evidence of a real commitment to evidence, and much evidence of a passion for branding. I hope this will change, but I would not bet on it.

Related Articles