Reflections on Gender Analyses of Bibliographic Corpora

Abstract

The interplay between an academic's gender and their scholarly output is a riveting topic at the intersection of scientometrics, data science, gender studies, and sociology. Its effects can be studied to analyze the role of gender in research productivity, tenure and promotion standards, collaboration and networks, or scientific impact, among others. The typical methodology in this field of research is based on a number of assumptions that are customarily not discussed in detail in the relevant literature, but undoubtedly merit a critical examination. Presumably the most confronting aspect is the categorization of gender. An author's gender is typically inferred from their name, further reduced to a binary feature by an algorithmic procedure. This and subsequent data processing steps introduce biases whose effects are hard to estimate. In this report we describe said problems and discuss the reception and interplay of this line of research within the field. We also outline the effect of obstacles, such as non-availability of data and code for transparent communication. Building on our research on gender effects on scientific publications, we challenge the prevailing methodology in the field and offer a critical reflection on some of its flaws and pitfalls. Our observations are meant to open up the discussion around the need and feasibility of more elaborated approaches to tackle gender in conjunction with analyses of bibliographic sources.

Authorship in top-ranked mathematical and physical journals: Role of gender on self-perceptions and bibliographic evidence

“…Despite these (and other) critiques, we have performed a namebased gender inference because academia is notoriously not gender agnostic and because gender disparities are indeed observed and need to be explained. We have discussed various concerns related to AGR in Mihaljević, Tullney, et al (2019) and would welcome ideas towards more inclusive schemas, preferably based on self-identification. Those would allow fairer, sustainable, and statistically significant analyses of bibliographic corpora in terms of gender.…”

“Why MANtoring is not the solution. A Rebuttal to ‘The association between early career informal mentorship in academic collaborations and junior author performance.’”

“…Second, like much of the literature on gender in science, the gender analysis by AlShebli et al engages in "Trans erasure" by failing to measure or thoughtfully acknowledge gender alternatives to cis men and cis women. These authors are not the first or only ones that fail to approach gender analyses in an inclusive way, which we and others have noted as a problem elsewhere 18,19 . We encourage scholars to both acknowledge the limitations of existing gender classification algorithms and strive to develop new measures, analyses, algorithms, and methods to include rather than erase trans and non-binary people.…”

Gender Gap in Natural Language Processing Research: Disparities in Authorship and Citations

“…A larger list of ethical considerations associated with the NLP Scholar project is available through the project webpage. 21 Mihaljević et al (2019) also discusses the ethical considerations in using author names to infer gender statistics in the Gender Gap in Science Project. 22…”

“…We discuss ethical considerations further in Section 6. See also Mihaljević et al (2019) for a discussion on ethical considerations in using author name to estimate gender statistics in the Gender Gap in Science Project-a large ongoing project tracking gender gaps in Mathematical and Natural Sciences. 4 Most studies on gender and authorship have found substantial gender disparities in favor of male researchers.…”