Independent discussion sections for improving inferential reproducibility in published research

Abstract

There is a reproducibility crisis in science. There are many potential contributors to replication failure in research across the translational continuum. In this perspective piece, we focus on the narrow topic of inferential reproducibility. Although replication of methods and results is necessary to demonstrate reproducibility, it is not sufficient. Also fundamental is consistent interpretation in the Discussion section. Current deficiencies in the Discussion sections of manuscripts might limit the inferential reproducibility of scientific research. Lack of contextualisation using systematic reviews, overinterpretation and misinterpretation of results, and insufficient acknowledgement of limitations are common problems in Discussion sections; these deficiencies can harm the translational process. Proposed solutions include eliminating or not reading Discussions, writing accompanying editorials, and post-publication review and comments; however, none of these solutions works very well. A second Discussion written by an independent author with appropriate expertise in research methodology is a new testable solution that could help probe inferential reproducibility, and address some deficiencies in primary Discussion sections.

Hypnotic depth and postoperative death: a Bayesian perspective and an Independent Discussion of a clinical trial

“…It has been suggested that an independent Discussion for scientific manuscripts can be useful in objectively assessing inferential reproducibility. 5 In Table 1, we provide a structured independent Discussion for the study by Sieber and colleagues, 3 and we comment on the similarities and differences between the original Discussion and the independent Discussion as an example of this approach. The author of the Independent Discussion (PEV) had access only to the manuscript's Introduction, Methods and Results, but not to the Abstract or the Discussion.…”

Combatting myths and misinformation about obstetric anesthesia

“…24 To address these biases, new peer-review strategies have attracted journals' interest, including double-blind peer review in which authors and their affiliations are concealed from reviewers, open peer review which reveals the full identities of authors and reviewers, and an 'independent' discussion section in which an independent expert who is not an author writes a second discussion. 24,25 These strategies may help to limit poor-quality research entering the public domain. In addition, we strongly recommend that journal editors of non-anesthesia journals obtain peer review from at least one expert obstetric anesthesiologist.…”

Gene edits to ‘CRISPR babies’ might have shortened their life expectancy

“…So, for some papers, the journal now asks independent specialists to write their own discussion. Unlike conventional peer reviewers, they look only at the methods and results sections and are blinded to the paper's conclusions 3 . The two discussions are published together, with similarities and differences highlighted.…”