Historically, authors in the biomedical field have often conflated the terms sex and gender in their research significantly limiting the reproducibility of the reported results. In the present study, we investigated current reporting practices around gender in biomedical publications that claim the identification of “gender differences”. Our systematic research identified 1117 articles for the year 2019. After random selection of 400 publications and application of inclusion criteria, 302 articles were included for analysis. Using a systematic evaluation grid, we assessed the provided methodological detail in the operationalization of gender and the provision of gender-related information throughout the manuscript. Of the 302 articles, 69 (23%) solely addressed biological sex. The remaining articles investigated gender, yet only 15 (6.5%) offered reproducible information about the operationalization of the gender dimension studied. Followingly, these manuscripts also provided more detailed gender-specific background, analyses and discussions compared to the ones not detailing the operationalization of gender. Overall, our study demonstrated persistent inadequacies in the conceptual understanding and methodological operationalization of gender in the biomedical field. Methodological rigor correlated with more nuanced and informative reporting, highlighting the need for appropriate training to increase output quality and reproducibility in the field.