An interactive dashboard to track themes, development maturity, and global equity in clinical artificial intelligence research

Zhang, Joe; Whebell, Stephen; Gallifant, Jack; Budhdeo, Sanjay; Mattie, Heather; Lertvittayakumjorn, Piyawat; Lopez, M.D.P. Arias; Tiangco, B.; Gichoya, Judy Wawira; Ashrafian, Hutan; Celi, Leo Anthony; Teo, James

doi:10.1016/s2589-7500(22)00032-2

Cited by 37 publications

(38 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Increasing attention is being paid to translational aspects of clinical AI 27 . Recent frameworks 28 and maturity classifications in literature reviews 1 , 29 adopt a high-level view of where an algorithm sits in its development roadmap. These supplement checklists for risk of bias and reporting that are internal to algorithm training and evaluation, for prediction 30 and diagnostic accuracy 31 , which focus on model-building 32 and generalisability 33 .…”

Section: Outside Of the Algorithmmentioning

confidence: 99%

“…There is a paucity of AI research in low to low-middle income countries (LLMIC) 1 , where there is also significant lack of diagnostic resource. Vertical integration promotes local infrastructure – a pre-requisite for representative data and implementation environments.…”

Section: Papsai - Vertical Integration In Response To Resource Scarcitymentioning

confidence: 99%

“…In many other cases, substantial infrastructural changes are unnecessary, as deployment requirements are lower. For example, the dominance of radiomics in development maturity 1 and devices 8 , may reflect lower implementation requirements from standardized data (DICOM) and pre-existing assisted reporting environments. In addition, the components we describe are not in themselves novel.…”

Section: Challenges and Solutions In Vertical Integrationmentioning

confidence: 99%

“…Multiple indicators over the past five years demonstrate accelerating interest in the application of artificial intelligence (AI) to human health, including exponentially increasing published research since 2016 1 , increasing healthcare provider interest in AI solutions 2 , 3 , soaring investment into AI startups 4 , 5 , and year-on-year increases in regulatory approvals 6 , 7 . In contrast, widespread translation of AI research into implementation remains conspicuously absent, particularly when considering AI for clinical decision-making, diagnosis, or prediction 8 .…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Moving towards vertically integrated artificial intelligence development

et al. 2022

Self Cite

View full text Add to dashboard Cite

Substantial interest and investment in clinical artificial intelligence (AI) research has not resulted in widespread translation to deployed AI solutions. Current attention has focused on bias and explainability in AI algorithm development, external validity and model generalisability, and lack of equity and representation in existing data. While of great importance, these considerations also reflect a model-centric approach seen in published clinical AI research, which focuses on optimising architecture and performance of an AI model on best available datasets. However, even robustly built models using state-of-the-art algorithms may fail once tested in realistic environments due to unpredictability of real-world conditions, out-of-dataset scenarios, characteristics of deployment infrastructure, and lack of added value to clinical workflows relative to cost and potential clinical risks. In this perspective, we define a vertically integrated approach to AI development that incorporates early, cross-disciplinary, consideration of impact evaluation, data lifecycles, and AI production, and explore its implementation in two contrasting AI development pipelines: a scalable “AI factory” (Mayo Clinic, Rochester, United States), and an end-to-end cervical cancer screening platform for resource poor settings (Paps AI, Mbarara, Uganda). We provide practical recommendations for implementers, and discuss future challenges and novel approaches (including a decentralised federated architecture being developed in the NHS (AI4VBH, London, UK)). Growth in global clinical AI research continues unabated, and introduction of vertically integrated teams and development practices can increase the translational potential of future clinical AI projects.

show abstract

Section: Outside Of the Algorithmmentioning

confidence: 99%

Section: Papsai - Vertical Integration In Response To Resource Scarcitymentioning

confidence: 99%

Section: Challenges and Solutions In Vertical Integrationmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Moving towards vertically integrated artificial intelligence development

et al. 2022

Self Cite

View full text Add to dashboard Cite

show abstract

“…Clinical artificial intelligence (AI) is a growing focus in academia, industry, and governments [ 1 - 3 ]. However, patients have benefited only in a few real-world contexts, reflecting a know-do gap called the “AI chasm” [ 4 , 5 ].…”

Section: Introductionmentioning

confidence: 99%

Stakeholder Perspectives of Clinical Artificial Intelligence Implementation: Systematic Review of Qualitative Evidence

Hogg¹,

Al-Zubaidy²,

Talks³

et al. 2023

J Med Internet Res

View full text Add to dashboard Cite

Background The rhetoric surrounding clinical artificial intelligence (AI) often exaggerates its effect on real-world care. Limited understanding of the factors that influence its implementation can perpetuate this. Objective In this qualitative systematic review, we aimed to identify key stakeholders, consolidate their perspectives on clinical AI implementation, and characterize the evidence gaps that future qualitative research should target. Methods Ovid-MEDLINE, EBSCO-CINAHL, ACM Digital Library, Science Citation Index-Web of Science, and Scopus were searched for primary qualitative studies on individuals’ perspectives on any application of clinical AI worldwide (January 2014-April 2021). The definition of clinical AI includes both rule-based and machine learning–enabled or non–rule-based decision support tools. The language of the reports was not an exclusion criterion. Two independent reviewers performed title, abstract, and full-text screening with a third arbiter of disagreement. Two reviewers assigned the Joanna Briggs Institute 10-point checklist for qualitative research scores for each study. A single reviewer extracted free-text data relevant to clinical AI implementation, noting the stakeholders contributing to each excerpt. The best-fit framework synthesis used the Nonadoption, Abandonment, Scale-up, Spread, and Sustainability (NASSS) framework. To validate the data and improve accessibility, coauthors representing each emergent stakeholder group codeveloped summaries of the factors most relevant to their respective groups. Results The initial search yielded 4437 deduplicated articles, with 111 (2.5%) eligible for inclusion (median Joanna Briggs Institute 10-point checklist for qualitative research score, 8/10). Five distinct stakeholder groups emerged from the data: health care professionals (HCPs), patients, carers and other members of the public, developers, health care managers and leaders, and regulators or policy makers, contributing 1204 (70%), 196 (11.4%), 133 (7.7%), 129 (7.5%), and 59 (3.4%) of 1721 eligible excerpts, respectively. All stakeholder groups independently identified a breadth of implementation factors, with each producing data that were mapped between 17 and 24 of the 27 adapted Nonadoption, Abandonment, Scale-up, Spread, and Sustainability subdomains. Most of the factors that stakeholders found influential in the implementation of rule-based clinical AI also applied to non–rule-based clinical AI, with the exception of intellectual property, regulation, and sociocultural attitudes. Conclusions Clinical AI implementation is influenced by many interdependent factors, which are in turn influenced by at least 5 distinct stakeholder groups. This implies that effective research and practice of clinical AI implementation should consider multiple stakeholder perspectives. The current underrepresentation of perspectives from stakeholders other than HCPs in the literature may limit the anticipation and management of the factors that influence successful clinical AI implementation. Future research should not only widen the representation of tools and contexts in qualitative research but also specifically investigate the perspectives of all stakeholder HCPs and emerging aspects of non–rule-based clinical AI implementation. Trial Registration PROSPERO (International Prospective Register of Systematic Reviews) CRD42021256005; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=256005 International Registered Report Identifier (IRRID) RR2-10.2196/33145

show abstract

Papers on artificial intelligence in healthcare

2024

How to Read a Paper

View full text Add to dashboard Cite

An interactive dashboard to track themes, development maturity, and global equity in clinical artificial intelligence research

Cited by 37 publications

References 9 publications

Moving towards vertically integrated artificial intelligence development

Moving towards vertically integrated artificial intelligence development

Stakeholder Perspectives of Clinical Artificial Intelligence Implementation: Systematic Review of Qualitative Evidence

Papers on artificial intelligence in healthcare

Contact Info

Product

Resources

About