Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension

Abstract

The CONSORT 2010 statement provides minimum guidelines for reporting randomized trials. Its widespread use has been instrumental in ensuring transparency in the evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate impact on health outcomes. The CONSORT-AI (Consolidated Standards of Reporting Trials–Artificial Intelligence) extension is a new reporting guideline for clinical trials evaluating interventions with an AI component. It was developed in parallel with its companion statement for clinical trial protocols: SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 29 candidate items, which were assessed by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a two-day consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The CONSORT-AI extension includes 14 new items that were considered sufficiently important for AI interventions that they should be routinely reported in addition to the core CONSORT 2010 items. CONSORT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention is integrated, the handling of inputs and outputs of the AI intervention, the human–AI interaction and provision of an analysis of error cases. CONSORT-AI will help promote transparency and completeness in reporting clinical trials for AI interventions. It will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the quality of clinical trial design and risk of bias in the reported outcomes.

Attitudes of the Surgical Team Toward Artificial Intelligence in Neurosurgery: International 2-Stage Cross-Sectional Survey

“…Furthermore, many members of the surgical team expressed motivation to participate in future AI adoption and research. To this end, frameworks of surgical device and technology innovation, such as The Idea, Development, Exploration, Assessment, Long-term study 20 and CONSORT-AI, 40 must be used to ensure transparent and robust translation of preclinical innovation into clinical practice. This will facilitate alleviating concerns of patients and the surgical team but also ensure we, as a community, are adhering to the ethical principles of modern medicine.…”

Key Principles of Clinical Validation, Device Approval, and Insurance Coverage Decisions of Artificial Intelligence

“…On the other hand, therapeutic or prophylactic actions can also bring about adverse reactions in some patients. Therefore, it is crucial to directly assess the effect of AI on clinical outcomes apart from its performance [ 40 ]. Such an evaluation is called validation of ‘clinical utility.’ Utility and efficacy are not interchangeable terms.…”

Going on up to the SPIRIT in AI: will new reporting guidelines for clinical trials of AI interventions improve their rigour?

“…On September 9, 2020, the SPIRIT-AI and CONSORT-AI extensions were published simultaneously in Nature Medicine, the BMJ, and Lancet Digital Health [4,5], with authors including regulators (FDA and MHRA) and editors of many of the leading medical journals. The hope is that the guidelines will better position journal editors, peer reviewers, and journal readers, who might be expert in clinical research or medical practice but less informed about AI to know what questions to ask of a manuscript in this field, to spot what is missing (whether intentional or not), to be better equipped to evaluate the quality of a study, and to make decisions based on its results.…”