Evaluation of Journal Registration Policies and Prospective Registration of Randomized Clinical Trials of Nonregulated Health Care Interventions

Abstract

IMPORTANCE Many interventions that are important to the health care of patients are not subject to regulation by the US Food and Drug Administration (FDA) or comparable regulatory bodies in other nations. OBJECTIVE To determine whether specialty journals that publish trials of primarily nonregulated health care interventions require prospective registration and whether the prospective registration policies are associated with the publication of prospectively registered trials, trials with adequately registered outcomes, and trials with primary outcomes consistent with the registered primary outcomes. DESIGN AND METHODS PubMed was searched daily, from March 18, 2016, to September 17, 2016, for nonregulated intervention randomized clinical trials. The search included all journals in the Clarivate Analytics Science Citation Index Expanded categories of behavioral sciences, nursing, nutrition and dietetics, psychology, rehabilitation, and surgery. Trials of interventions not subject to FDA regulation were included. One investigator extracted journal registration policy and trial registration status. Two investigators independently extracted trial registration and publication characteristics. MAIN OUTCOMES AND MEASURES For journals, the main outcome was the trial registration policy. For trials, the main outcomes were prospective registration, adequacy of outcome registration, and concordance of registered with published primary outcomes. RESULTS In total, 953 nonregulated intervention trials published in 254 journals were identified. Prospective registration was required for publication by 29 (11.4%) of 254 journals, and an additional 12 journals (4.7%) had conditional date-based requirements. Only 189 (19.8%) of the 953 trials were registered prospectively, including 33 of 98 published in journals with prospective registration policies as compared with 156 of 855 in journals without policies (33.7% vs 18.2%; P = .004). Among the 17 journals that required prospective registration and had at least 2 included trials, none had a prospective registration of more than 50%. In journals with policies, only 3 of 98 trials included primary outcomes consistent with prospectively, adequately registered outcomes, as compared with 34 of 852 trials in journals without policies (3.1% vs 4.0%; P = .62). CONCLUSIONS AND RELEVANCE Few journals in behavioral sciences or psychology, nursing, nutrition and dietetics, rehabilitation, and surgery require prospective trial registration, and those with existing registration policies rarely enforce them; this finding suggests that strategies for encouraging prospective registration of clinical trials not subject to FDA regulation should be developed and tested.

Commentary on Vassar et al. (2020): Cautionary observations on the pre‐registration revolution

“…The paper by Vassar et al . , along with findings from other academic disciplines , show that we are still a long way from resolving the credibility or reproducibility crisis that exists in many fields of empirical research. Specifically, these studies highlight the limited inroads made by pre‐registration, one of the main tools available to academic journals to improve the quality and integrity of published research.…”

Mapping outcome reporting bias in published non-pharmacological randomized trials: a protocol for a cross-sectional study

“…In this sense, an outcome may be defined using all following elements: 1 -outcome domain, 2 -specific measurement (or measure), 3 -specific metric, 4 -method of aggregation, and 5 -time point (or time frame) (12). For example, a domain can be: depression; a specific measure: Beck Depression Inventory score; a specific assessment time point: 12-month post randomization; a specific metric: change from baseline or final value; and a method of aggregation: mean or proportion (13). It can be considered a partially defined outcome one that has a domain, a measure and a time point (9,13,14), i.e., a minimum number of items is needed to define an outcome required by international registry platforms (7).…”

“…For example, a domain can be: depression; a specific measure: Beck Depression Inventory score; a specific assessment time point: 12-month post randomization; a specific metric: change from baseline or final value; and a method of aggregation: mean or proportion (13). It can be considered a partially defined outcome one that has a domain, a measure and a time point (9,13,14), i.e., a minimum number of items is needed to define an outcome required by international registry platforms (7). US Department of Health and Human services instructs for all Applicable Clinical Trials (ACTs), specially the funded by NIH, to report the specific measure (or measurement) and time point for each outcome at registration, reflecting the current international standards.…”

“…Because of this, an outcome may be considered only fully specified when all 5 elements are present (10). Given the five possible elements that constitute an outcome and a difference in criteria required by international standards (3 elements), it can be considered an adequately reported outcome one that is partially defined (with a domain, measure and time point) or even fully specified (if all 5 possible elements are present) being defined in the registry in a retrospective manner (6,8,9,(11)(12)(13)(14)(15)(16). In this example, "mean change from baseline of Beck Depression Inventory score 12-month post randomization", stated in the primary outcome section in a registry (done prospectively -before the initiation of the clinical trial) in clinicaltrials.gov platform (or other platform) with a registry number (13), would be considered a fully specified outcome (10) and hence adequately reported (13).…”

“…To our knowledge, the frequency of how common selective outcome reporting and trial registration occur in non-pharmacological interventions is still scarce. Problems in outcomes have been studied to this date considering non-pharmacological interventions in separate of each other (19)(20)(21)(22)(23)(24)(25), and also some studies that analyzed this in a more systematic way had other focuses rather than analyzing the reporting of outcomes specifically (13,19,26,27). Non-pharmacological interventions may include exercise, physical therapy, diet, nutrition, manual therapy, psychotherapy, rehabilitation, complementary and alternative medicine (28)(29)(30) constituting approximately 40% of clinical trials in the literature, and the number of these interventions will continue to raise in the future (31).…”