Interpretable Trials: Is Interpretability a Reason Why Clinical Trials Fail?

Abstract

Background: There are clinical trials using composite measures, indices, or scales as proxy for independent variables or outcomes. Interpretability of derived measures may not be satisfying. Adopting indices of poor interpretability in clinical trials may lead to trial failure. This study aims to understand the impact of using indices of different interpretability in clinical trials.Methods: The interpretability of indices was categorized as: fair-to-poor, good, and unknown. In the literature, frailty indices were considered fair to poor interpretability. Body mass index (BMI) was highly interpretable. The other indices were of unknown interpretability. The trials were searched at clinicaltrials.gov on October 2, 2018. The use of indices as conditions/diseases or other terms was searched. The trials were grouped as completed, terminated, active, and other status. We tabulated the frequencies of frailty, BMI, and other indices.Results: There were 263,928 clinical trials found and 155,606 were completed or terminated. Among 2,115 trials adopting indices or composite measures as condition or disease, 244 adopted frailty and 487 used BMI without frailty indices. Significantly higher proportions of trials of unknown status used indices as conditions/diseases or other terms, compared to completed and terminated trials. The proportions of active trials using frailty indices were significantly higher than those of completed or terminated trials.Discussion: Clinical trial databases can be used to understand why trials may fail. Based on the findings, we suspect that using indices of poor interpretability may be associated with trial failure. Interpretability has not been conceived as an essential criterion for outcomes or proxy measures in trials. We will continue verifying the findings in other databases or data sources and apply this research method to improve clinical trial design. To prevent patients from experiencing trials likely to fail, we suggest further examining the interpretability of the indices in trials.

Mental Illness Diagnoses May Not Cause All Mental Symptoms: A Simulation Study for Major Depressive Episodes, Dysthymic Disorder, and Manic Episodes

“…Third, recent evidence suggests that the use of composite diagnostic criteria of inferior interpretability, i.e., frailty indices, may be associated with early trial termination or failure [ 33 ]. For mental health researchers, it is important not only to ensure the conditions in their trials aligned with the Research Domain Criteria well [ 34 ], but also to assess the interpretability of the diagnoses.…”

Critical Hierarchical Appraisal and repOrting tool for composite measureS (CHAOS)

“…In this tool, we suggested using well-established statistics, such as R-squared, regression coefficients, and pseudo-R-squared, to interpret composite measures by regressing composite measures on input variables (Question 13) and bias variables or measurement errors (Question 14). In our study, analyzing the interpretability of indices or composite measures in clinical trials, composite measures that are not likely well interpreted by their input variables, is associated with early termination of clinical trials [26].…”

Diagnostic accuracy of symptoms for an underlying disease: a simulation study

“…Therefore, the diagnoses of frailty have poor relationships with the input symptoms and do not predict major outcomes better than their input symptoms 1 . When tested in trials, the use of the diagnoses of poor interpretability, such as frailty, is associated with early termination of trials 32 .…”