Real-world post-deployment performance of a novel machine learning-based digital health technology for skin lesion assessment and suggestions for post-market surveillance

Thomas, Lucy; Hyde, Chris; Mullarkey, Dan; Greenhalgh, Jack; Kalsi, Dilraj; Ko, Justin

doi:10.3389/fmed.2023.1264846

Cited by 7 publications

(3 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…By location, the two studies [46,47] published by Australian authors reported comparatively lower sensitivity values in CNNs than previously reported [39][40][41][43][44][45]48,49]. Australia remains the nation with the highest incidence of melanoma [2].…”

Section: Performance Of Bedside Cnn Versus Cliniciansmentioning

confidence: 82%

“…There were 11 articles [39][40][41][42][43][44][45][46][47][48][49] which reported performance on bedside CNNs. One article's [42] main outcome was monitoring changes in sequential digital dermatoscopy and presented results differently and could not be compared.…”

Section: Assessment Of Performance Metrics Cnnmentioning

confidence: 99%

“…There was a high degree of variance between studies, which may be a result of a multitude of factors, including the number of participants, the demographics (not always included in the studies), the type of CNN compared (mobile, bedside and TBP) and the type of MM detected (number of in situ vs. invasive). This may be, in part, a measure of the variance of geographic locations of these studies, with six countries (Australia [46], Canada [44], Germany [39,43], Switzerland [37], UK [48,49] and USA [34]) recording studies.…”

Section: Performance Of Bedside Cnn Versus Cliniciansmentioning

confidence: 99%

See 2 more Smart Citations

Performance of Commercial Dermatoscopic Systems That Incorporate Artificial Intelligence for the Identification of Melanoma in General Practice: A Systematic Review

Miller,

Rosic,

Stapelberg

et al. 2024

Cancers

View full text Add to dashboard Cite

Background: Cutaneous melanoma remains an increasing global public health burden, particularly in fair-skinned populations. Advancing technologies, particularly artificial intelligence (AI), may provide an additional tool for clinicians to help detect malignancies with a more accurate success rate. This systematic review aimed to report the performance metrics of commercially available convolutional neural networks (CNNs) tasked with detecting MM. Methods: A systematic literature search was performed using CINAHL, Medline, Scopus, ScienceDirect and Web of Science databases. Results: A total of 16 articles reporting MM were included in this review. The combined number of melanomas detected was 1160, and non-melanoma lesions were 33,010. The performance of market-approved technology and clinician performance for classifying melanoma was highly heterogeneous, with sensitivity ranging from 16.4 to 100.0%, specificity between 40.0 and 98.3% and accuracy between 44.0 and 92.0%. Less heterogeneity was observed when clinicians worked in unison with AI, with sensitivity ranging between 83.3 and 100.0%, specificity between 83.7 and 87.3%, and accuracy between 86.4 and 86.9%. Conclusion: Instead of focusing on the performance of AI versus clinicians for classifying melanoma, more consistent performance has been obtained when clinicians’ work is supported by AI, facilitating management decisions and improving health outcomes.

show abstract

Section: Performance Of Bedside Cnn Versus Cliniciansmentioning

confidence: 82%

Section: Assessment Of Performance Metrics Cnnmentioning

confidence: 99%

Section: Performance Of Bedside Cnn Versus Cliniciansmentioning

confidence: 99%

See 1 more Smart Citation

Performance of Commercial Dermatoscopic Systems That Incorporate Artificial Intelligence for the Identification of Melanoma in General Practice: A Systematic Review

Miller,

Rosic,

Stapelberg

et al. 2024

Cancers

View full text Add to dashboard Cite

show abstract

Artificial Intelligence Applications for Imaging Metabolic Bone Diseases

Isaac,

Akdogan,

Dalili

et al. 2024

Semin Musculoskelet Radiol

View full text Add to dashboard Cite

Artificial intelligence (AI) has significantly impacted the field of medical imaging, particularly in diagnosing and managing metabolic bone diseases (MBDs) such as osteoporosis and osteopenia, Paget's disease, osteomalacia, and rickets, as well as rare conditions such as osteitis fibrosa cystica and osteogenesis imperfecta. This article provides an in-depth analysis of AI techniques used in imaging these conditions, recent advancements, and their clinical applications. It also explores ethical considerations and future perspectives. Through comprehensive examination and case studies, we highlight the transformative potential of AI in enhancing diagnostic accuracy, improving patient outcomes, and contributing to personalized medicine. By integrating AI with existing imaging techniques, we can significantly enhance the capabilities of medical imaging in diagnosing, monitoring, and treating MBDs. We also provide a comprehensive overview of the current state, challenges, and future prospects of AI applications in this crucial area of health care.

show abstract

A service evaluation of the acceptability of DERM to patients and staff at three Community Diagnostic Hubs: study protocol for interviews

Hunt,

Grigore,

Javan-Noughabi

et al. 2024

Preprint

View full text Add to dashboard Cite

Background More rapid and efficient diagnosis of suspected skin cancers has been a target for improvement in the NHS for several years. DERM (Deep Ensemble for the Recognition of Melanoma) is a Class IIa UKCA-Certified Artificial Intelligence Medical Device which has been suggested could have a role in achieving this. Implementation projects have confirmed this where DERM is used to filter referrals which have already been made to hospital (post-referral clinical pathway). Attention has now turned to the possible role of the technology earlier in the clinical pathway, in community diagnostic hubs. This protocol describes plans to assess how acceptable DERM is to patients and staff, and how satisfied patients and staff are with the service experienced in DERM skin cancer community diagnostic hubs. It is part of a wider service evaluation designed to answer the question: “What standard does this service achieve?” which also includes evaluation of the safety, effectiveness, and cost-effectiveness on introducing DERM. Methods This protocol details qualitative interview methods nested within a broader mixed methods study. Using responses to a background patient/health care worker satisfaction questionnaire as a starting point, up to30 patients and staff will be invited to take part in online qualitative interviews across three implementation sites. We will aim to include patients where photography has been deemed appropriate and patients have consented to take part on the DERM pathway. Data will be analysed thematically using framework analysis and summarised narratively. Discussion In combination with parallel investigations exploring DERM accessibility, test accuracy, effectiveness via a controlled before and after study, and cost-effectiveness, this service evaluation aims to describe the standard achieved by DERM used in community diagnostic hubs. This will in turn inform any further roll-out of the service identifying aspects for improvement and identifying barriers to adoption.

show abstract

Real-world post-deployment performance of a novel machine learning-based digital health technology for skin lesion assessment and suggestions for post-market surveillance

Cited by 7 publications

References 31 publications

Performance of Commercial Dermatoscopic Systems That Incorporate Artificial Intelligence for the Identification of Melanoma in General Practice: A Systematic Review

Performance of Commercial Dermatoscopic Systems That Incorporate Artificial Intelligence for the Identification of Melanoma in General Practice: A Systematic Review

Artificial Intelligence Applications for Imaging Metabolic Bone Diseases

A service evaluation of the acceptability of DERM to patients and staff at three Community Diagnostic Hubs: study protocol for interviews

Contact Info

Product

Resources

About