Honest signaling in academic publishing

Abstract

Academic journals provide a key quality-control mechanism in science. Yet, information asymmetries and conflicts of interests incentivize scientists to deceive journals about the quality of their research. How can honesty be ensured, despite incentives for deception? Here, we address this question by applying the theory of honest signaling to the publication process. Our models demonstrate that several mechanisms can ensure honest journal submission, including differential benefits, differential costs, and costs to resubmitting rejected papers. Without submission costs, scientists benefit from submitting all papers to high-ranking journals, unless papers can only be submitted a limited number of times. Counterintuitively, our analysis implies that inefficiencies in academic publishing (e.g., arbitrary formatting requirements, long review times) can serve a function by disincentivizing scientists from submitting low-quality work to high-ranking journals. Our models provide simple, powerful tools for understanding how to promote honest paper submission in academic publishing.

Competition for priority harms the reliability of science, but reforms can help

“…When negative and positive results are equally valuable, scientists have no incentive to conduct large studies in order to increase the probability of positive results-a large study costs more time but generates the same payoff as an underpowered study that can quickly produce a negative result. How does this finding fit into ongoing discussions about whether researchers should publish all of their results 2,53,54 , or whether some types of publication bias 5,39,55,56 or publication restrictions 10,[57][58][59] are desirable? Our model points out that, when study quality is not sufficiently rewarded, a bias against negative results incentivizes scientists to conduct larger studies.…”

“…Note that the mechanism by which start-up costs incentivize higher-quality research does not necessarily depend on the timing of such costs. Any costs disproportionately paid by scientists who attempt to conduct quick, low-quality research will serve the same function 10 . These might include wrap-up costs, such as long peer review times, or costs at other points in the research pipeline (for example, an obligation to peer review n other papers for each submitted paper).…”

“…However, in recent years, scientists have begun to examine the utility of even longstanding characteristics of this institution 1-3 . For example, it is now widely recognized that preferentially valuing positive over negative results can generate publication bias, which distorts the published literature 4,5 ; evaluating scientists based on their number of publications can cause a myopic focus on productivity at the expense of rigour 6 ; and rewarding scientists based on the prestige of the journal in which they publish may incentivize scientists to present their work in an overly positive light, submit low-quality papers to high-impact journals and engage in other questionable research practices [7][8][9][10][11][12] .The priority rule is a particularly longstanding scientific norm, in which individuals who are first to make discoveries receive disproportionate credit relative to all other individuals who provide solutions to the same problem 13,14 . Famously, Charles Darwin was motivated to publish his writings on evolution by means of natural selection in part because of a concern that he would lose priority to Alfred Russel Wallace, who had developed a similar idea.…”

“…However, in recent years, scientists have begun to examine the utility of even longstanding characteristics of this institution 1-3 . For example, it is now widely recognized that preferentially valuing positive over negative results can generate publication bias, which distorts the published literature 4,5 ; evaluating scientists based on their number of publications can cause a myopic focus on productivity at the expense of rigour 6 ; and rewarding scientists based on the prestige of the journal in which they publish may incentivize scientists to present their work in an overly positive light, submit low-quality papers to high-impact journals and engage in other questionable research practices [7][8][9][10][11][12] .…”

Mechanistic Modeling for the Masses

“…The lack of specificity and resulting lack of alignment between theory and statistical model may be an instance of deceptive signaling (in the ecological sense, which doesn't imply intent to deceive), where a lack of theoretical rigor is covered up with statistical tests and a recitation of related observed correlations. This maps the generalizability crisis onto analogous problems for which models already exist as starting points, including models of signaling in collaborative environments (Smaldino, Flamson, & McElreath, 2018;Smaldino & Turner, 2020;Tiokhin et al, 2021), the evolution of scientific knowledge on networks (Zollman, 2007;2010;2013;O'Connor & Weatherall, 2018;, and the effect of prevailing social power on individual choices (Henrich & Boyd, 2008;Higginson & Munafò, 2016;Bergstrom, Foster, & Song, 2016;O'Connor, 2019). With some further development, these models could be used to conduct several "what if things are different" computational experiments under a variety of assumptions to understand what might happen if various interpersonal or institutional changes were instituted.…”