Do women write better code than men?
Researchers at Cal Poly San Luis Obispo and North Carolina University analysed data from 1.4 million users of the web-based Git repository hosting service, Github, and found that computer code written by women received a higher approval rating (on average) compared to men.
The researchers looked at the data of users who logged in to the service on April 1, 2015.
While GitHub doesn’t ask users for gender information, the researchers were able to determine gender based on information from user profiles or by matching email addresses to Google+ profiles.
Out of 4,037,953 GitHub user profiles with email addresses, the team was able to identify 1,426,121 (35.3%) of them as men or women through their public Google+ profiles – the team believes it is the first use of this technique in a large scale study.
Pull requests – proposed changes to a software project’s code – made by women on Github were more likely to be accepted than suggestions made by men. They found that 78.6% of pull requests made by women were accepted compared with 74.6% of those by men.
It should be noted that the paper is currently awaiting peer review, meaning that other experts still need to evaluate the quality of the study.
The researchers were aware that the gender linking approach raised privacy concerns, but took several steps to ensure protection of identities.
They said in the paper: “First, this research has undergone human subjects IRB review, research that is based entirely on publicly available data. Second, we have informed Google about our approach to determine whether they believe that it’s a privacy violation of their users to be able to link email address to gender; they responded that it’s consistent with Google’s terms of service. Third, to protect the identities of the people described in this study to the extent possible, we do not plan to release our data that links GitHub users to genders.”
The study identified a clear gender bias
Interestingly, profiles of users who made it clear that they were female had a much lower acceptance rate than female users who didn’t provide gender information in their profile.
The researchers said that their findings identified a clear gender bias.
“For outsiders, we see evidence for gender bias: women’s acceptance rates are 71.8% when they use gender neutral profiles, but drop to 62.5% when their gender is identifiable. There is a similar drop for men, but the effect is not as strong,” the paper noted.
“Women have a higher acceptance rate of pull requests overall, but when they’re outsiders and their gender is identifiable, they have a lower acceptance rate than men.
“Our results suggest that although women on Github may be more competent overall, bias against them exists nonetheless,” the researchers concluded.
Citation: PeerJ PrePrints – “Gender Bias in Open Source: Pull Request Acceptance of Women Versus Men”, Josh Terrell, Andrew Kofink, Justin Middleton, Clarissa Rainear, Emerson Murphy-Hill, Chris Parnin
https://doi.org/10.7287/peerj.preprints.1733v1 | CC-BY 4.0 Open Access | rec: 9 Feb 2016, publ: 9 Feb 2016