This study explores how researchers’ analytical choices affect the reliability of scientific findings. Most discussions of reliability problems in science focus on systematic biases. We broaden the lens to emphasize the idiosyncrasy of conscious and unconscious decisions that researchers make during data analysis. We coordinated 161 researchers in 73 research teams and observed their research decisions as they used the same data to independently test the same prominent social science hypothesis: that greater immigration reduces support for social policies among the public. In this typical case of social science research, research teams reported both widely diverging numerical findings and substantive conclusions despite identical start conditions. Researchers’ expertise, prior beliefs, and expectations barely predict the wide variation in research outcomes. More than 95% of the total variance in numerical results remains unexplained even after qualitative coding of all identifiable decisions in each team’s workflow. This reveals a universe of uncertainty that remains hidden when considering a single study in isolation. The idiosyncratic nature of how researchers’ results and conclusions varied is a previously underappreciated explanation for why many scientific hypotheses remain contested. These results call for greater epistemic humility and clarity in reporting scientific findings.

Women continue to earn less than their male counterparts globally. Scholars and feminist activists have suggested a partial explanation for this gender gap in earnings could be women's limited access to power structures at the workplace. Using the linked employer–employee data of the Workplace Employment Relations Study 2004–2011, this article asks what happens to the gender gap in earnings among non‐managerial employees when the share of women in management at the workplace increases. The findings, based on workplace‐fixed time‐fixed effects regression models, suggest that workplace‐level increases in the share of women in management are associated with decreases of the non‐managerial gender gap in earnings. This effect appears to be largely unrelated to changes in equality and diversity policies, family‐friendly arrangements and support for carers at the workplace.

In an influential article published in the American Sociological Review in 2009, Herring finds that diverse workforces are beneficial for business. His analysis supports seven out of eight hypotheses on the positive effects of gender and racial diversity on sales revenue, number of customers, perceived relative market share, and perceived relative profitability. This comment points out that Herring’s analysis contains two errors. First, missing codes on the outcome variables are treated as substantive codes. Second, two control variables—company size and establishment size—are highly skewed, and this skew obscures their positive associations with the predictor and outcome variables. We replicate Herring’s analysis correcting for both errors. The findings support only one of the original eight hypotheses, suggesting that diversity is nonconsequential, rather than beneficial, to business success.

Replicating published studies promotes active learning of quantitative research skills. Drawing on experiences from a replication course, we provide practical tips and reflections for teachers who consider incorporating replication in their courses. We discuss teaching practices and challenges we encountered at three stages of a replication course: student recruitment, course structure and proceedings, and learning outcomes. We highlight that by engaging in replication, students learn from established scholarly work in a collaborative and reflective manner. Students not only improve their quantitative literacy but also learn more generally about the scientific method and the production of research.

This article answers several related questions: does parenthood affect whether women hold positions of authority? Is there a parenthood effect on authority for men? Is the gender gap in authority explained by a more deleterious effect of parenthood on women’s in comparison to men’s representation in positions of authority? Past studies of the relationship between parenthood and workplace authority have been limited in their ability to assess a causal effect of parenthood because most have employed a static approach, measuring the presence of children and the type of job held concurrently, using cross-sectional data. Using retrospective life course data from four rounds of the Family Survey of the Dutch Population and distributed fixed-effects models, we study within-person changes in having supervisory authority among women and men in the years before, around, and after the birth of their first child. The findings show a moderate negative effect of motherhood on women’s representation in authority, which is entirely explained by a reduction in the number of hours worked. Fatherhood has no effect on men’s representation in authority. The gender gap in supervisory authority between women and men grows over time but is already very large years before the transition to first-time parenthood.