Charting the Constellation of Science Reform

Abstract

Over the past decade, a sense of urgency has been building in the scientific community. They have discovered that much of the literature body is unreliable and possibly invalid thanks to weak theory, flawed methods, and shoddy statistics. This is driven by a widespread competitive, secretive approach to research, which, in turn, is fueled by toxic academic incentive structures. Many in the community have decided to address these issues, coming together in what has become known as the ‘scientific reform movement’. While these ‘reformers’ are often spoken of a single, homogeneous entity, my findings underscore the heterogeneity of the reform community. In my dissertation, I explore the scientific reform group using ethnography and social network analysis tools. I primarily studied their online Twitter engagements to understand their culture, practices, and structure. With Wenger’s Community of Practice theory as an interpretive framework, I analyze scientific reform discourse playing out between reformers on Twitter. Using quantitative Twitter friend/follow data, I investigate which reform members engage online, using following behavior to understand aspects of their social structure. I link the quantitative exploration with my qualitative analysis, to conclude that while the reformers are united by their interest in improving science, they are better characterized as a constellation of small communities of practice, each with their own norms, priorities, and unique approach to the group enterprise of scientific reform. My investigation is an exercise in reflexivity as I have studied a community in which I am an active part.

Questionable Metascience Practices

“…In discussing QMPs, we should be careful not to homogenize metascience (Field, 2022) or to presume that QMPs are necessarily problematic. It is likely that only some metascientists engage in some QMPs some of the time and that QMPs are only problematic in some situations.…”

“…In addition, and at the risk of homogenizing metascience (Field, 2022), some (not all) metascientists focus their concerns on particular aspects of "the scientific method" (Flis, 2019). In particular, the contemporary metascientific view of science tends to focus on:…”

The Misalignment of Incentives in Academic Publishing and Implications for Journal Reform

“…This means that PCI is likely to be used by those with limited financial resources, in addition to other members of the community. Finally, some subgroups in the science reform movement are strongly invested in accessibility, openness, and inclusivity (33). They may want to support the PCI model, by contributing to and by using it, on principle as much for its utility.…”

The societal impact of Open Science–a scoping review

“…The adoption of Open Science (OS) policies by research institutions and funders globally aligns with the aim of fostering societal impact. OS is both a set of practices and a research reform movement that aims to make academic research (henceforth, 'research') more transparent, inclusive and accessible (8). It includes diverse practices like Open Access (OA) to research publications and Open and FAIR Data; the creation and use of Open Code and Software; practising process transparency through pre-registration and Open Methods; and evaluation transparency through Open Evaluation (e.g., Open Peer Review); and Citizen Science (CS), co-creation, participatory research, and collaboration (9,10).…”