Lessons Learned About Autonomous AI: Finding a Safe, Efficacious, and Ethical Path Through the Development Process

Current Challenges and Barriers to Real-World Artificial Intelligence Adoption for the Healthcare System, Provider, and the Patient

“…Within the realm of design, AI has been based upon maximally reducible characteristics aligned with the scientific knowledge of human clinician cognition, rather than proxy characteristics. 13 , 14 With regard to appropriate data usage, AI creators now must collect data in compliance with regulations and legislation, as well as maximum traceability from the data pedigree, and steward the data accordingly. 15 , 16 To maximize alignment among clinical workflow, evidence-based clinical standards of care, and practice patterns from the quality of care organizations, professional medical societies and patient organizations are expressing their views and establishing preferred practice patterns.…”

“… 21 Lastly the assignment of liability or other protections is being defined based on the accountability principle for the autonomous AI output commensurate with indications. 13 Last year, the AMA included in its AI policy that autonomous AI creators are responsible and liable if any harm should be caused by the diagnostic system they create. 13 , 22 , 23 This is most pertinent for autonomous AI, where it would be ill-suited to expect a provider to be held liable for a diagnosis that is out of scope and comfort level from their usual practice and expertise.…”

Protecting Data Privacy in the Age of AI-Enabled Ophthalmology

“…Two additional bioethical principles-non-maleficence (often translated as "first, do no harm") and respect for autonomy-are also considered central to biomedical ethics and AI development. 17,18 The Belmont Report is not directly applicable to secondary use of de-identified clinical data; in the report, when identifying information has been removed, the use of that data is no longer considered human subjects research. Nonetheless, its core principles are instructive.…”

Approaching autonomy in medical artificial intelligence

“…The level of AI autonomy is broadly accepted to be crucial to the risk-assessment of AI algorithms, yet its definition and classification is poorly defined. [2][3][4] In this Comment, we present a nuanced discussion of autonomy in medical AI to improve the understanding of the effect, risks, and clinical value of such systems.…”

“…2,9,10 Autonomous models that are intended to diagnose, treat, or reduce disease risk should be evaluated in preregistered, prospective studies, with value determined on the basis of clinical outcomes and not simply comparison to clinician performance. 3 Given the black box nature of many AI algorithms and concerns regarding generalisability, such technologies should first undergo safety testing as assistive technologies before being evaluated as autonomous algorithms. Rigorous outcome-based testing with performance metrics that address safety, efficacy, and equity during clinical evaluation is necessary for ethical system development.…”