Influence of peer review on the reporting of primary outcome(s) and statistical analyses of randomised trials

Abstract

BackgroundSelective reporting of outcomes in clinical trials is a serious problem. We aimed to investigate the influence of the peer review process within biomedical journals on reporting of primary outcome(s) and statistical analyses within reports of randomised trials.MethodsEach month, PubMed (May 2014 to April 2015) was searched to identify primary reports of randomised trials published in six high-impact general and 12 high-impact specialty journals. The corresponding author of each trial was invited to complete an online survey asking authors about changes made to their manuscript as part of the peer review process. Our main outcomes were to assess: (1) the nature and extent of changes as part of the peer review process, in relation to reporting of the primary outcome(s) and/or primary statistical analysis; (2) how often authors followed these requests; and (3) whether this was related to specific journal or trial characteristics.ResultsOf 893 corresponding authors who were invited to take part in the online survey 258 (29%) responded. The majority of trials were multicentre (n = 191; 74%); median sample size 325 (IQR 138 to 1010). The primary outcome was clearly defined in 92% (n = 238), of which the direction of treatment effect was statistically significant in 49%. The majority responded (1–10 Likert scale) they were satisfied with the overall handling (mean 8.6, SD 1.5) and quality of peer review (mean 8.5, SD 1.5) of their manuscript. Only 3% (n = 8) said that the editor or peer reviewers had asked them to change or clarify the trial’s primary outcome. However, 27% (n = 69) reported they were asked to change or clarify the statistical analysis of the primary outcome; most had fulfilled the request, the main motivation being to improve the statistical methods (n = 38; 55%) or avoid rejection (n = 30; 44%). Overall, there was little association between authors being asked to make this change and the type of journal, intervention, significance of the primary outcome, or funding source. Thirty-six percent (n = 94) of authors had been asked to include additional analyses that had not been included in the original manuscript; in 77% (n = 72) these were not pre-specified in the protocol. Twenty-three percent (n = 60) had been asked to modify their overall conclusion, usually (n = 53; 88%) to provide a more cautious conclusion.ConclusionOverall, most changes, as a result of the peer review process, resulted in improvements to the published manuscript; there was little evidence of a negative impact in terms of post hoc changes of the primary outcome. However, some suggested changes might be considered inappropriate, such as unplanned additional analyses, and should be discouraged.Electronic supplementary materialThe online version of this article (doi:10.1186/s13063-017-2395-4) contains supplementary material, which is available to authorized users.

A meta-research study revealed several challenges in obtaining placebos for investigator-initiated drug trials

“…Second, the response rate in the survey was only 26%. Although this is comparable to other online surveys [17,18] and the sample seems to be representative in terms of measured baseline characteristics, we cannot know if the investigators who responded constitute a self-selected sample experiencing more problems compared to nonresponders. Third, the sample size was relatively small.…”

Outcome reporting discrepancies between trial entries and published final reports of orthodontic randomized controlled trials

“…Moreover, the influence of peer review and editorial processes on the level of selective reporting was not recorded, as the final manuscript was only assessed. However, there is evidence that peer review itself possibly tends to have limited effect on the delineation of outcomes (19) with verification of outcomes reliant on the availability of registry entries or trial protocols to peer reviewers. Consequently, novel approaches to alleviate selective reporting may well be required.…”

Enhancing Social Performance of Ngos Operation in Indonesia Through External Positive Pressure: Mediating Role of Orientation Dimensions and Sustainable Practices

“…This study is majorly based on exploring the impact of external pressure on the social performance of an organization. In order to justify the tested hypothesis, the online survey-based statistical outcomes are considered for critical evaluation (Hopewell et al, 2018;Leavy, 2017;Silverman, 2016). The quantitative research method is used in the data collection mechanism where majorly those participants are considered who belongs to Indonesian NGOs and organizational community individuals.…”