Cross-sectional analysis of the reporting of continuous outcome measures and clinical significance of results in randomized trials of non-pharmacological interventions

Abstract

BackgroundReporting the scoring details of continuous outcome measures in randomized trials allows readers to interpret the size of any effect of the intervention. This study aimed to determine, in a sample of randomized trials: 1) the completeness of reporting of scoring details for continuous outcome measures, and 2) whether trial authors comment on the clinical significance of statistically significant trial results.MethodsA descriptive analysis of randomized trials of non-pharmacological interventions published during 2009 in the six leading general medical journals (n = 138), and which used at least one continuous outcome measure (n = 85). From each trial report, two authors independently extracted the following information about each continuous outcome measure: the reporting of its scoring details, presentation of its results, and the reporting and justification of the clinical significance of the results.ResultsAcross the 84 trials, we identified 336 continuous outcome measures. A total of 146 (44%) were published measures, 12 (4%) were adapted from published measures, 5 (1%) were developed for the trial, and 173 (51%) were ‘conventional measures’ for which scoring details are not necessary (such as weight). For 57 (35%) of the 163 non-conventional outcome measures no scoring details or reference to the outcome measure were provided in the trial report. Of the 159 outcome measures with a statistically significant result, clinical significance was not mentioned for 81 (51%) and was reported without any elaboration or justification for 39 (25%) of them.ConclusionsScoring details of continuous outcome measures used in this sample of randomized trials of non-pharmacological interventions were incompletely reported, which hampers interpretation of a trial’s results. Complete reporting of scoring details is important when considering the clinical significance of the results. When deciding about an intervention, having this information may help clinicians in their conversations with patients about the possible benefits and harms, and their size, of the intervention.Electronic supplementary materialThe online version of this article (doi:10.1186/1745-6215-15-362) contains supplementary material, which is available to authorized users.

Rehabilitation interventions in randomized controlled trials for low back pain: proof of statistical significance often is not relevant

“…Our results are coherent with the literature where the reporting of results in terms of clinical relevance is sparsely used across trials [1]. We confirm the preliminary results published by Van Tulder et al focused on exercise therapy for chronic LBP reporting that less than half of studies (39%) with positive conclusions shown clinically important differences [11].…”

“…We call for more adherence to reporting of planned sample size including the clinical relevance with the clinical interpretation of the effects. Without the complete information, the reader is unable to fully interpret the results of a study [1]. On one hand, authors have to report all elements used for sample size calculation, including the clinical relevance.…”

“…Randomized controlled trials (RCTs) aim to show differences in an outcome measurement between two or more groups of patients undergoing different interventions [1]. Authors of RCTs usually report findings in term of statistical significance (i.e., with a p -value < 0.05 the intervention is more effective than the comparison).…”

Clinical significance in pediatric oncology randomized controlled treatment trials: a systematic review

“…The minority (4.2%) of randomized questions explicitly identified the delta value as the MCID with justification. These findings are in line with the limited number of previous studies investigating clinical significance reporting, wherein under-reporting was found [ 13 , 19 , 20 , 28 , 29 ]. Chan et al [ 13 ] found that, among a random sample of 27 RCTs in major medical journals, 20 articles included sample size calculations, 90% of which reported a delta value but only 11% stated that the delta value was chosen to reflect the MCID of the intervention.…”

“…Research suggests that RCT authors rely primarily on statistical significance, they do not consistently provide their own interpretation of the clinical importance of results, and they rarely provide sufficient information to enable readers to draw their own conclusions [ 13 , 19 , 20 ]. The degree to which clinical significance has been assessed in the pediatric oncology literature remains unknown.…”