The orthopaedic trauma literature: an evaluation of statistically significant findings in orthopaedic trauma randomized trials

Abstract

Background: Evidence-based medicine posits that health care research is founded upon clinically important differences in patient centered outcomes. Statistically significant differences between two treatments may not necessarily reflect a clinically important difference. We aimed to quantify the sample sizes and magnitude of treatment effects in a review of orthopaedic randomized trials with statistically significant findings.

Epidemiology and incidence of tibia fractures in the Swedish Fracture Register

“…Quality registers also enable scientific assessments in areas for which randomised, controlled trials are not always possible [9] . When the absolute risk of complications is low, quality registers are able to detect crucial differences, while randomised, controlled trials may not include enough patients to do this [10,11] . Another crucial role for the National Quality Registers (NQR) is to be hypothesis generating for subsequent RCTs.…”

Role of Technology Assessment in Orthopaedics

“…Even some of the Level I evidence may be suspect. For example, the quality of randomized controlled trials of nonpharmacological interventions (eg, surgery, arthroscopy) for hip and knee osteoarthritis is less than that of trials of pharmacological interventions [11], and smaller randomized controlled trials in the orthopaedic literature tend to find larger effects than larger ones, rendering the results of these smaller trials suspect [41]. That only about half of all systematic reviews take quality into account in their analysis and interpretation of results or in their discussion sections [31] highlights the need for critical reading of assessments and reviews, as does the fact that there can be a subjective component to interpreting results [38].…”

Everyone (Else) is Conflicted

“…Multiple clinical trials on a study question should be encouraged and widely published, especially those that challenge a previous trial. This is especially true in orthopaedic surgery, where average trial sizes are small; studies in our literature often enroll fewer than 100 patients, and so are extremely prone to Type II (false negative) or spurious Type I (false positive) errors [13]. These small trials, which often report few total outcome events are highly correlated with very large and generally implausible treatment effects [13].…”

“…This is especially true in orthopaedic surgery, where average trial sizes are small; studies in our literature often enroll fewer than 100 patients, and so are extremely prone to Type II (false negative) or spurious Type I (false positive) errors [13]. These small trials, which often report few total outcome events are highly correlated with very large and generally implausible treatment effects [13]. Studies with fewer than 50 total outcome events that favor a new drug or device should be viewed with considerable skepticism until confirmed by other independently conducted clinical trials.…”