The Scientific Impact of Positive and Negative Phase 3 Cancer Clinical Trials

Abstract

Importance Positive Phase III cancer clinical trials are widely hailed, while negative trials are often interpreted as scientific failures. We hypothesized that these interpretations would be reflected in the scientific literature. Objective To compare the scientific impact of positive versus negative phase III cancer clinical treatment trials. Design We examined the phase III trial history of SWOG over a 30-year period (1985–2014). Scientific impact was assessed according to multiple publication and citation outcomes. Citation data were obtained using Google Scholar™. Citation counts were compared using generalized estimating equations for Poisson regression. Setting SWOG, a national cancer clinical trials consortium. Participants Any trial that was formally evaluated for the randomized treatment comparison was included for analysis of publication and citation outcomes. Exposures Trials were categorized as positive if they achieved a statistically significant result in favor of the new treatment for the protocol-specified primary endpoint, and negative otherwise. Main Outcomes The main outcomes were the impact factors for the journals publishing the primary trial results, and the number of citations for the primary trial manuscripts and all secondary manuscripts associated with the trials. Results Ninety-four studies enrolling n=46,424 patients were analyzed. Twenty-eight percent of trials were positive (26/94). The primary publications from positive trials were published in journals with higher average impact factors (28.4 vs. 17.5, p=.007) and were cited twice as often as negative trials (average per year, 43 vs. 21, p=.03). However, the number of citations from all primary and secondary manuscripts did not differ between positive and negative trials (average per year, 55 vs. 45, p=.53). Conclusions and Relevance The scientific impact of the primary manuscripts from positive phase III randomized cancer clinical trials was twice as great as for negative trials. But when all of the manuscripts associated with the trials were considered, the scientific impact between positive and negative trials was similar. Positive trials indicate clinical advances, but negative trials also have a sizeable scientific impact by generating important scientific observations and new hypotheses and by showing what new treatments should not be used.

The Role of Clinical Trial Participation in Cancer Research: Barriers, Evidence, and Strategies

“…The predominant reason that trials fail to complete is poor accrual. 126 These failures represent a lost investment on the part of funding agencies. Moreover, when clinical trials close because of failure to accrue, non-financial costs are also incurred.…”

“…Informed consent documents rare rarely include the risk of closure due to lack of study participation, despite the fact that about 1 in 4 randomized phase III trials have just such an outcome. 126 …”

The Effect of Positive SWOG Treatment Trials on Survival of Patients With Cancer in the US Population

“…The model did not reflect the value of negative trials. 54 As one example, many thousands of lives have likely been saved based on randomized clinical trials showing that autologous bone-marrow transplant for breast cancer – a treatment which had achieved wide dissemination in the cancer treatment community despite lack of conclusive evidence in trials – in fact did not work. 55–57 Finally, the total research costs (as opposed to the phase III trial costs alone) per life year gained for any new drug is inevitably higher, since the conduct of a phase III comparative trial represents the end of the life cycle of treatment development, which also includes drug discovery, development and early stage trials.…”

Identification of Multiple Cryptococcal Fungicidal Drug Targets by Combined Gene Dosing and Drug Affinity Responsive Target Stability Screening

“…It was disappointing that bithionol itself was not a more effective anticryptococcal agent. We felt that it was important to include negative data concerning this drug for CNS cryptococcosis, both to prevent inadvertent clinical use and to address recent concerns that negative data are sometimes omitted despite their potentially sizeable scientific impact in generating new hypotheses ( 40 ). Serum and brain drug levels of approximately 40 and 1 µg/ml, respectively, corresponded to levels that produced only a 50% fungicidal activity in our assay, suggesting that ineffective absorption and transport through the blood-brain barrier was a strong contributor to this ineffectiveness in the mouse model.…”