Statistical significance: p value, 0.05 threshold, and applications to radiomics—reasons for a conservative approach

Abstract

Here, we summarise the unresolved debate about p value and its dichotomisation. We present the statement of the American Statistical Association against the misuse of statistical significance as well as the proposals to abandon the use of p value and to reduce the significance threshold from 0.05 to 0.005. We highlight reasons for a conservative approach, as clinical research needs dichotomic answers to guide decision-making, in particular in the case of diagnostic imaging and interventional radiology. With a reduced p value threshold, the cost of research could increase while spontaneous research could be reduced. Secondary evidence from systematic reviews/metaanalyses, data sharing, and cost-effective analyses are better ways to mitigate the false discovery rate and lack of reproducibility associated with the use of the 0.05 threshold. Importantly, when reporting p values, authors should always provide the actual value, not only statements of "p < 0.05" or "p ≥ 0.05", because p values give a measure of the degree of data compatibility with the null hypothesis. Notably, radiomics and big data, fuelled by the application of artificial intelligence, involve hundreds/thousands of tested features similarly to other "omics" such as genomics, where a reduction in the significance threshold, based on well-known corrections for multiple testing, has been already adopted.

A decade of radiomics research: are images really data or just patterns in the noise?

“…To further add to this complexity, in most cases, complex statistical approaches and machine learning are used to build prediction models based on radiomics features which come with their own challenges [ 12 , 13 ]. To tackle these issues and avoid getting lost in a reproducibility crisis, careful methodological and statistical consideration of potential pitfalls is crucial [ 7 , 14 ].…”

Computed tomography-derived myocardial extracellular volume: an early biomarker of cardiotoxicity in esophageal cancer patients undergoing radiation therapy

“…Statistical analysis was performed with R v3.5.3. P values < 0.05 were considered as significant [17].…”

Lean body weight versus total body weight to calculate the iodinated contrast media volume in abdominal CT: a randomised controlled trial

“…To appraise homogeneity between the two groups, preliminary comparisons were performed for baseline characteristics, including the distribution of the injected ICM volume. A p value of less than 0.05 [ 38 ] indicated a significant difference between the groups.…”