Industry ties and evidence in public comments on the FDA framework for modifications to artificial intelligence/machine learning-based medical devices: a cross sectional study

Abstract

ObjectivesTo determine the extent and disclosure of financial ties to industry and use of scientific evidence in comments on a US Food and Drug Administration (FDA) regulatory framework for modifications to artificial intelligence/machine learning (AI/ML)-based software as a medical device (SaMD).DesignCross-sectional study.SettingWe searched all publicly available comments on the FDA ‘Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD)—Discussion Paper and Request for Feedback’ from 2 April 2019 to 8 August 2019.Main outcome measuresThe proportion of articles submitted by parties with financial ties to industry, disclosing those ties, citing scientific articles, citing systematic reviews and meta-analyses, and using a systematic process to identify relevant literature.ResultsWe analysed 125 comments submitted on the proposed framework. 79 (63%) comments came from parties with financial ties; for 36 (29%) comments, it was not clear and the absence of financial ties could only be confirmed for 10 (8%) comments. No financial ties were disclosed in any of the comments that were not from industry submitters. The vast majority of submitted comments (86%) did not cite any scientific literature, just 4% cited a systematic review or meta-analysis and no comments indicated that a systematic process was used to identify relevant literature.ConclusionsFinancial ties to industry were common and undisclosed, and scientific evidence, including systematic reviews and meta-analyses, were rarely cited. To ensure regulatory frameworks best serve patient interests, the FDA should mandate disclosure of potential conflicts of interest (including financial ties) in comments, encourage the use of scientific evidence, and encourage engagement from non-conflicted parties.

Artificial intelligence and hybrid imaging: the best match for personalized medicine in oncology

“…AI-based software follow the same regulation of medical devices (Pesapane et al 2018 ) but their specific feature (i.e., learning and adapting to new data in real time to increase performance) has raised concerns about the suitability of traditional medical device regulatory pathways resulting in a FDA’s proposal to modify this regulatory framework (FDA 2019 ). Until 8 August 2019, more than 100 comments were submitted in response to the proposed regulatory framework, but they lacked in scientific evidence (only 15 comments cited at least 1 paper—systematic review/meta-analysis in only 5 cases—published in an academic journal) while resulting certainly related to parties with financial ties in about two third of the cases (Smith et al 2020 ). Obviously, financial ties represents a conflict of interest and, as such, it should also be disclosed in the regulatory pathways as typically occurs in medical science, since it may, even not necessarily, lead to biased commenting (Smith et al 2020 ).…”

“…Until 8 August 2019, more than 100 comments were submitted in response to the proposed regulatory framework, but they lacked in scientific evidence (only 15 comments cited at least 1 paper—systematic review/meta-analysis in only 5 cases—published in an academic journal) while resulting certainly related to parties with financial ties in about two third of the cases (Smith et al 2020 ). Obviously, financial ties represents a conflict of interest and, as such, it should also be disclosed in the regulatory pathways as typically occurs in medical science, since it may, even not necessarily, lead to biased commenting (Smith et al 2020 ). Disclosure is an important first step in improving transparency and reducing privacy concerns especially when it deals to sensitive data such as in healthcare even if it will not necessarily alter the interpretation of information (Pesapane et al 2018 ; Smith et al 2020 ).…”

“…Obviously, financial ties represents a conflict of interest and, as such, it should also be disclosed in the regulatory pathways as typically occurs in medical science, since it may, even not necessarily, lead to biased commenting (Smith et al 2020 ). Disclosure is an important first step in improving transparency and reducing privacy concerns especially when it deals to sensitive data such as in healthcare even if it will not necessarily alter the interpretation of information (Pesapane et al 2018 ; Smith et al 2020 ).…”

Artificial Intelligence and Machine Learning in Prostate Cancer Patient Management—Current Trends and Future Perspectives

“…Regulation is mandatory for every test that will be introduced in clinical practice. The latest regulatory papers in the European Union and the United States of America will undergo certification beyond self-validation and certification studies to prove the reproducible result of the test and to avoid its nonreproducible risk [ 96 , 97 ].…”

Are Regulations Safe? Reflections From Developing a Digital Cancer Decision-Support Tool

“…The presence of a commercial bias was evidenced by a 2020 study showing that the public discourse around a regulatory framework for software as a medical device proposed by US Food and Drug Administration lacked scientific support and commonly involved undisclosed financial ties with industry. 67 Despite these criticisms, loss-transfer approaches are rational choices when it is more difficult to predict the behavior of a complex system than it is to manage its undesirable consequences. A dynamic approach to risk management that acknowledges a prediction horizon, multiple levels, dependencies, and adaptations is thus needed.…”