Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension

Abstract

The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human-AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret, and critically appraise the design and risk of bias for a planned clinical trial.

Artificial Intelligence Applications for COVID-19 in Intensive Care and Emergency Settings: A Systematic Review

“…Studies developing AI models should adhere to the TRIPOD reporting guidelines 22 , PROBAST 23 , or, ideally, recent AI-specific guidelines. These include the guidelines for transparency, reproducibility, ethics, and effectiveness (TREE) 42 , CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence) 43 , and SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) 44 . While the above guidelines provide comprehensive explanations and elaborations, we emphasise hereinafter several common problematic areas within the reviewed studies and recommendations for future studies.…”

Reporting guidelines for clinical trials of artificial intelligence interventions: the SPIRIT-AI and CONSORT-AI guidelines

“…To address this, the SPIRIT-AI and CONSORT-AI Steering Group announced in October 2019 an initiative to develop evidencebased extensions for clinical trial protocols and reports involving AI interventions [19]. The SPIRIT-AI [20][21][22] and CONSORT-AI [23][24][25] guidelines have since been developed in accordance with the EQUATOR (Enhancing the Quality and Transparency of Health Research) Network recommendations. The guidelines were developed using a Delphi methodology with an international multidisciplinary consortium.…”

“…The SPIRIT-AI [20][21][22] and CONSORT-AI [23][24][25] guidelines include 15 and 14 new items, respectively, that should be routinely reported in addition to the core SPIRIT 2013 [13] and CONSORT 2010 [14] items. For example, the new guidelines recommend that investigators should provide clear descriptions of the AI intervention, including instructions and skills required for use, the study setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human-AI interaction, and the analysis of error cases.…”

“…The SPIRIT-AI [20][21][22] and CONSORT-AI [23][24][25] guidelines have the potential to improve the quality of clinical trials of AI systems in health, through improvements in design and delivery, and the completeness and transparency of their reporting. It is, however, important to appreciate the context in which these guidelines sit.…”

“…It is, however, important to appreciate the context in which these guidelines sit. First, the new items within SPIRIT-AI [20][21][22] and CONSORT-AI [23][24][25] are all extensions or elaborations rather than substitutes for the core items. Core considerations addressed by SPIRIT 2013 [13] and CONSORT 2010 [14] remain important in all clinical trial protocols and reports, regardless of the intervention itself.…”