Statistics in publishing: the (mis)use of the p-value (part 1)

Statistics in publishing: the (mis)use of the p-value (Part 2)

“…In our example, we then conducted the study using a sample from the population of patients and analysed the gathered data. This resulted in a p-value of 0.07 as shown in Table 1 of Part 1 of Broekstra et al (2022a), and most researchers would then conclude that there was no significant difference in functional outcome between both interventions. In our experience, many would follow this procedure routinely, without understanding exactly the underlying line of reason.…”

“…Given the drawbacks of NHST, there is a need for a simple, alternative way of drawing inferences from the data of our studies. One that we have already alluded to in Part 1 of Broekstra et al (2022a), and that is known to most researchers, is the confidence interval (CI). This is a measure of precision of the estimated effect: the narrower the CI, the more precise the estimated effect.…”

“…In our example study with the distal radial fracture patients (Broekstra et al., 2022a), the p -value of 0.07 gives the probability of observing a difference between the two groups of 1.21 points in terms of the Patient-Rated Wrist Evaluation (PRWE) score between both study groups, when we assumed that there was no difference in efficacy of both treatments in the entire population of patients. When the probability of an observed, or more extreme observation under the assumption that H 0 is true (i.e.…”

“…Moreover, a difference of 2 points is more compatible with the data than a difference of 0 points, which illustrates that just dismissing the intervention because ‘0’ is in the interval is nonsensical. Finally, as discussed in Part 1 (Broekstra et al., 2022a), it does make sense to take into account the minimal clinically important change (MCIC). Now, we can do that using the CI.…”

“…This is the second of a two-part article, in which we discuss problems associated with null hypothesis significance testing (NHST) and the use of the p -value. In the first part, we described how NHST, and the p- value can be misused, the potential consequences and offered some practical solutions (Broekstra et al., 2022a). To illustrate this, we used a simulated dataset based on a randomized controlled trial (RCT) that investigated women with a distal radial fracture, randomized either to an intervention (cast + rehabilitation programme) or control (cast only) group (Broekstra et al., 2022b).…”