Toward safer ophthalmic artificial intelligence via distributed validation on real-world data

Abstract

Purpose of review The current article provides an overview of the present approaches to algorithm validation, which are variable and largely self-determined, as well as solutions to address inadequacies. Recent findings In the last decade alone, numerous machine learning applications have been proposed for ophthalmic diagnosis or disease monitoring. Remarkably, of these, less than 15 have received regulatory approval for implementation into clinical practice. Although there exists a vast pool of structured and relatively clean datasets from which to develop and test algorithms in the computational ‘laboratory’, real-world validation remains key to allow for safe, equitable, and clinically reliable implementation. Bottlenecks in the validation process stem from a striking paucity of regulatory guidance surrounding safety and performance thresholds, lack of oversight on critical postdeployment monitoring and context-specific recalibration, and inherent complexities of heterogeneous disease states and clinical environments. Implementation of secure, third-party, unbiased, pre and postdeployment validation offers the potential to address existing shortfalls in the validation process. Summary Given the criticality of validation to the algorithm pipeline, there is an urgent need for developers, machine learning researchers, and end-user clinicians to devise a consensus approach, allowing for the rapid introduction of safe, equitable, and clinically valid machine learning implementations.

Динамика Модели SIR: Взгляд На Эпидемии И Вакцинацию

“…However, there are challenges and methodologies involved in this process. One challenge is the limited availability of data in the early stages of an epidemic, which can hinder the performance of the model [Nath et al 2023]. Optimal resource allocation strategies during an epidemic have been discussed in several studies.…”

American Society of Retina Specialists Artificial Intelligence Task Force Report

“…Although there are many barriers to AI implementation in general, among which are real-world pre-deployment and post-deployment validation and monitoring, trust, and workflow issues, a core issue is the current paradigm of who designs the models and for which use. 50,51 Clinician-led model design, optimizing for the everyday needs of clinicians and patients, likely has a higher chance of actualizing real-world implementation in ophthalmology and meeting patient-relevant endpoints. However, there are several barriers to enabling the democratization of clinicianled AI.…”

The Chinese Clinical Sleep Database: An Innovative Database System Includes Large-Scale Clinical Data of Chinese Population

“… 15 Establishing a database with a large number of clinical samples may help developing models that meet clinical practice requirements and have high reliability. 16 , 17 …”

“…15 Establishing a database with a large number of clinical samples may help developing models that meet clinical practice requirements and have high reliability. 16,17 In this study, in collaboration with Adai Technology (Beijing) Ltd., Co, Beijing, China, we established a public sleep database -The Chinese Clinical Sleep Database (CCSD, psg.wangli-tech.com), 18 aiming to continuously collect and store the clinical data from routine clinical care and health monitoring processes through standardized procedures.…”