Statistical significance misuse in public health research: an investigation of the current situation and possible solutions

Abstract

Background. Despite the efforts of leading statistical authorities and experts worldwide and the inherent dangers of interpretative errors in clinical research, misuses of statistical significance remain a common practice in the field of public health. Currently, there is a need to attempt to quantify this phenomenon. Methods. 100 studies were randomly selected within the PubMed database. An evaluation system for the interpretation, presentation, and communication of results (IPC) was adopted, which provided for a maximum of 11 points and a minimum acceptability threshold of 5 points. Results. The median of the results was 2 points out of the available 11 (IQR = 1). The difference from the minimum acceptable IPC score of 5 was substantial (90|95|99-% CI: [2; 4]) and, assuming all the Wilcoxon test requirements have been sufficiently met, highly surprising at the statistical level (P-value < .001, S-value > 10). In total, 13 out of 100 studies achieved the minimum score of 5 points. Conclusion. These findings provide solid evidence of widespread and severe methodological shortcomings in the use of statistical significance measures in clinical and public health research during 2023. Therefore, it is essential for academic journals to compulsorily demand higher scientific quality standards.