Nonreplicable publications are cited more than replicable ones

Serra-García, Marta; Gneezy, Uri

doi:10.1126/sciadv.abd1705

Cited by 118 publications

(62 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Oppositely, the CPT, despite still being far from clinical pain, has a longer duration and reaches higher intensity (i.e., maximum tolerance), leading to a sensation that is a better proxy to real-life pain ( 20 , 21 ). In addition, given the ongoing “replication crisis” affecting natural sciences ( 52 , 53 ), the successful replication of our previous results on a different type of experimentally induced pain adds value to the current study.…”

Section: Discussionsupporting

confidence: 54%

The Temporal Modulation of Nocebo Hyperalgesia in a Model of Sustained Pain

et al. 2022

View full text Add to dashboard Cite

BackgroundThe direction and the magnitude of verbal suggestions have been shown to be strong modulators of nocebo hyperalgesia, while little attention has been given to the role of their temporal content. Here, we investigate whether temporal suggestions modulate the timing of nocebo hyperalgesia in an experimental model of sustained pain.MethodsFifty-one healthy participants were allocated to one of three groups. Participants received an inert cream and were instructed that the agent had either hyperalgesic properties setting in after 5 (Nocebo 5, N5) or 30 (Nocebo 30, N30) minutes from cream application, or hydrating properties (No Expectation Group, NE). Pain was induced by the Cold Pressure Test (CPT) which was repeated before cream application (baseline) and after 10 (Test10) and 35 (Test35) minutes. Changes in pain tolerance and in HR at each test point in respect to baseline were compared between the three groups.ResultsTolerance change at Test 10 (Δ10) was greater in N5 (MED = −36.8; IQR = 20.9) compared to NE (MED = −5.3; IQR = 22.4; p < 0.001) and N30 (MED = 0.0; IQR = 23.1; p < 0.001), showing that hyperalgesia was only present in the group that expected the effect of the cream to set in early. Tolerance change at Test 35 (Δ35) was greater in N5 (MED = −36.3; IQR = 35.3; p = 0.002) and in N30 (MED = −33.3; IQR = 34.8; p = 0.009) compared to NE, indicating delayed onset of hyperalgesia in N30, and sustained hyperalgesia in N5. No group differences were found for HR.ConclusionsOur study demonstrated that temporal expectations shift nocebo response onset in a model of sustained pain.

show abstract

Section: Discussionsupporting

confidence: 54%

The Temporal Modulation of Nocebo Hyperalgesia in a Model of Sustained Pain

et al. 2022

View full text Add to dashboard Cite

show abstract

“…Similarly, overestimation of expected utility gain can happen if research which we are more certain will not replicate tends to get cited more often. Recent research indicates the latter situation might often be the case (Serra-Garcia & Gneezy, 2021). Note that we could in principle repeat this thought experiment for any two variables in figure 4 not already joined by a causal arrow.…”

Section: Discussionmentioning

confidence: 76%

Replication value as a function of citation impact and sample size

Isager¹,

Veer²,

Lakens³

2021

Preprint

View full text Add to dashboard Cite

Researchers seeking to replicate original research often need to decide which of several relevant candidates to select for replication. Several strategies for study selection have been proposed, utilizing a variety of observed indicators as criteria for selection. However, few strategies clearly specify the goal of study selection and how that goal is related to the indicators that are utilized. We have previously formalized a decision model of replication study selection in which the goal of study selection is to maximize the expected utility gain of the replication e?ort. We further define the concept of replication value as a proxy for expected utility gain (Isager et al., 2020). In this article, we propose a quantitative operationalization of replication value. Wefirst discuss how value and uncertainty - the two concepts used to determine replication value – could be estimated via information about citation count and sample size. Second, we propose an equation for combining these indicators into an overall estimate of replication value, which we denote RVCn. Third, we suggest how RVCn could be implemented as part of a broader study selection procedure. Finally, we provide preliminary data suggesting that studies that were in fact selected for replication tend to have relatively high RVCn estimates. The goal of this article is to explain how RVCn is intended to work and, in doing so, demonstrate the many assumptions that should be explicit in any replication study selection strategy.

show abstract

“…This problem is escalated by the policies of top journals and many grant institutions, which always favor the use of the newest technology to answer a research question. An indication that method hopping contributes to replication failures is the fact that publications in high-profile journals such as Nature and Science that could not be replicated are cited 153 times more often than those studies that could be successfully replicated ( Serra-Garcia and Gneezy, 2021 ). It is suggested that more-cited papers have more “interesting” results driven by new technologies which leads to a subjectively more lax peer-review process and subsequently to a negative correlation between replicability and citation count ( Serra-Garcia and Gneezy, 2021 ).…”

Section: A Summary Of Reasons That Contribute To Replication Failures...mentioning

confidence: 99%

Ten Points to Improve Reproducibility and Translation of Animal Research

Spanagel

2022

Front. Behav. Neurosci.

View full text Add to dashboard Cite

Findings from animal experiments are often difficult to transfer to humans. In this perspective article I discuss two questions. First, why are the results of animal experiments often so difficult to transfer to humans? And second, what can be done to improve translation from animal experiments to humans? Translation failures are often the result of poor methodology. It is not merely the fact that low statistical power of basic and preclinical studies undermine a “real effect,” but the accuracy with which data from animal studies are collected and described, and the resulting robustness of the data is generally very low and often does not allow translation to a much more heterogeneous human condition. Equally important is the fact that the vast majority of publications in the biomedical field in the last few decades have reported positive findings and have thus generated a knowledge bias. Further contributions to reproducibility and translation failures are discussed in this paper, and 10 points of recommendation to improve reproducibility and translation are outlined. These recommendations are: (i) prior to planning an actual study, a systematic review or potential preclinical meta-analysis should be considered. (ii) An a priori power calculation should be carried out. (iii) The experimental study protocol should be pre-registered. (iv) The execution of the study should be in accordance with the most recent ARRIVE guidelines. (v) When planning the study, the generalizability of the data to be collected should also be considered (e.g., sex or age differences). (vi) “Method-hopping” should be avoided, meaning that it is not necessary to use the most advanced technology but rather to have the applied methodology under control. (vii) National or international networks should be considered to carry out multicenter preclinical studies or to obtain convergent evidence. (viii) Animal models that capture DSM-5 or ICD-11 criteria should be considered in the context of research on psychiatric disorders. (ix) Raw data of publication should be made publicly available and should be in accordance with the FAIR Guiding Principles for scientific data management. (x) Finally, negative findings should be published to counteract publication bias. The application of these 10 points of recommendation, especially for preclinical confirmatory studies but also to some degree for exploratory studies, will ultimately improve the reproducibility and translation of animal research.

show abstract

Nonreplicable publications are cited more than replicable ones

Cited by 118 publications

References 31 publications

The Temporal Modulation of Nocebo Hyperalgesia in a Model of Sustained Pain

The Temporal Modulation of Nocebo Hyperalgesia in a Model of Sustained Pain

Replication value as a function of citation impact and sample size

Ten Points to Improve Reproducibility and Translation of Animal Research

Contact Info

Product

Resources

About