Continuous Improvement of Digital Health Applications Linked to Real-World Performance Monitoring: Safe Moving Targets?

Gilbert, Stephen; Pimenta, Andreia; Stratton‐Powell, Ashley A.; Welzel, Cindy; Melvin, Tom

doi:10.1016/j.mcpdig.2023.05.010

Cited by 6 publications

(1 citation statement)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…AIeMDs represent a special case because adaptation based on learning is a basic principle of this technology. To address the role of adaptability in AI technologies, a continuous assessment is necessary, which could be realized by the implementation of predetermined change control plans into the proposed holistic system [2,[30][31][32].…”

mentioning

confidence: 99%

Holistic Human-Serving Digitization of Health Care Needs Integrated Automated System-Level Assessment Tools

Welzel,

Cotte,

Wekenborg

et al. 2023

J Med Internet Res

Self Cite

View full text Add to dashboard Cite

Digital health tools, platforms, and artificial intelligence– or machine learning–based clinical decision support systems are increasingly part of health delivery approaches, with an ever-greater degree of system interaction. Critical to the successful deployment of these tools is their functional integration into existing clinical routines and workflows. This depends on system interoperability and on intuitive and safe user interface design. The importance of minimizing emergent workflow stress through human factors research and purposeful design for integration cannot be overstated. Usability of tools in practice is as important as algorithm quality. Regulatory and health technology assessment frameworks recognize the importance of these factors to a certain extent, but their focus remains mainly on the individual product rather than on emergent system and workflow effects. The measurement of performance and user experience has so far been performed in ad hoc, nonstandardized ways by individual actors using their own evaluation approaches. We propose that a standard framework for system-level and holistic evaluation could be built into interacting digital systems to enable systematic and standardized system-wide, multiproduct, postmarket surveillance and technology assessment. Such a system could be made available to developers through regulatory or assessment bodies as an application programming interface and could be a requirement for digital tool certification, just as interoperability is. This would enable health systems and tool developers to collect system-level data directly from real device use cases, enabling the controlled and safe delivery of systematic quality assessment or improvement studies suitable for the complexity and interconnectedness of clinical workflows using developing digital health technologies.

show abstract

mentioning

confidence: 99%