Fostering reproducibility, reusability, and technology transfer in health informatics

Hauschild, Anne-Christin; Eick, Lisa; Wienbeck, Joachim; Heider, Dominik

doi:10.1016/j.isci.2021.102803

Cited by 7 publications

(4 citation statements)

References 33 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Here, to establish an academia-tailored software life cycle, we propose a comprehensive guideline for research facilities derived from the requirements of IEC 62304. 16 Complementary to our quality management guideline, 50 we propose to address these challenges by following our guideline, which lowers the barriers to a potential technology transfer toward the medical industry. Furthermore, in the supplemental information , we provide a comprehensive checklist for a successful Software Life Cycle and demonstrate the feasibility of our guideline with our implementation example.…”

Section: Discussionmentioning

confidence: 99%

Guideline for software life cycle in health informatics

Hauschild¹,

Martin²,

Wienbeck³

et al. 2022

iScience

Self Cite

View full text Add to dashboard Cite

Section: Discussionmentioning

confidence: 99%

Guideline for software life cycle in health informatics

Hauschild¹,

Martin²,

Wienbeck³

et al. 2022

iScience

Self Cite

View full text Add to dashboard Cite

“…Reproducibility, described as "the ability of an independent research team to produce the same results using the AI method based on the documentation made by the original research team" [15], requires an exact representation of all relevant aspects of the study development and realization. This includes the complete information of the used software and source code, the original data as well as the correct documentation of crucial details and precise instructions for the implementation [39][40][41]. The reproducibility in AI builds trust in the developed models and results [14,40].…”

Section: Contributions To Better Reproducibility In Ai In Medicine An...mentioning

confidence: 99%

“…The lack of proper upkeep of essential resources, such as data, code, or instructions, hinders advancements in research and impedes reproducibility [42]. In addition, the current academic environment encourages researchers to publish prototypes of their AI models rather than ensuring a fully verified system [41], which also impacts the quality of these models. Figure 5 illustrates the three important elements of medical AI research.…”

Section: Contributions To Better Reproducibility In Ai In Medicine An...mentioning

confidence: 99%

Guidelines and Standard Frameworks for Artificial Intelligence in Medicine: A Systematic Review

Shiferaw,

Roloff,

Balaur

et al. 2024

Preprint

View full text Add to dashboard Cite

A growing volume of evidence marks the potential of Artificial Intelligence (AI) in medicine, in improving diagnostic accuracy, clinical decision support, risk/event prediction, drug discovery, and patient management. However, the continuous integration of AI into clinical settings requires the development of up-to-date and robust guidelines and standard frameworks that consider the evolving challenges of AI implementation in medicine. This review evaluates these guidelines quality and summarizes ethical frameworks, best practices, and recommendations. The Appraisal of Guidelines, Research, and Evaluation (AGREE II) tool was used to assess the quality of guidelines based on six domains: scope and purpose, stakeholder involvement, rigor of development, clarity of presentation, applicability, and editorial independence. The protocol of this review including the eligibility criteria, the search strategy data extraction sheet and methods, was published prior to the actual review with International Registered Report Identifier (IRRID) of (DERR1-10.2196/47105). The initial search resulted in 4,975 studies from two databases and five studies from manual search. Nine articles were selected for data extraction based on the eligibility criteria. We found that while guidelines generally excel in scope, purpose, and editorial independence, there is significant variability in applicability and the rigour of guideline development. Well-established initiatives such as DECIDE-AI, SPIRIT-AI, and CONSORT-AI have shown high quality, particularly in terms of stakeholder involvement. However, applicability remains a prominent challenge among the guidelines. We conclude that the reproducibility, ethical and environmental aspects of AI in medicine still need attention from both medical and AI communities. This review emphasizes the crucial need for high-quality guidelines and opens a new avenue in evaluating guidelines themselves. Our work highlights the need for working toward the development of integrated and comprehensive reporting guidelines that adhere to the principles of Findability, Accessibility, Interoperability and Reusability (FAIR). This alignment is essential for fostering a cultural shift towards transparency and open science, which are pivotal milestone for sustainable digital health research.

show abstract

“…The reproducibility crisis leads to an enormous waste of resources but also to massive impediments to translation [36]. The flood of false-positive exploratory studies makes picking the right solutions for translation difficult.…”

Section: Reproducible Sciencementioning

confidence: 99%

Artificial Intelligence in Healthcare: Lost In Translation?

Madai,

Higgins

2021

Preprint

View full text Add to dashboard Cite

Artificial intelligence (AI) in healthcare is a potentially revolutionary tool to achieve improved healthcare outcomes while reducing overall health costs. While many exploratory results hit the headlines in recent years there are only few certified and even fewer clinically validated products available in the clinical setting. This is a clear indication of failing translation due to shortcomings of the current approach to AI in healthcare. In this work, we highlight the major areas, where we observe current challenges for translation in AI in healthcare, namely precision medicine, reproducible science, data issues and algorithms, causality, and product development. For each field, we outline possible solutions for these challenges. Our work will lead to improved translation of AI in healthcare products into the clinical setting. BackgroundHealthcare systems worldwide suffer from ageing populations and exploding medical costs [1]. On the current trajectory, they will soon become economically unsustainable. A possible solution is the application of artificial intelligence (AI) technology, which has the potential to enable improved healthcare outcomes and to reduce overall costs [2].This can be attributed to the main properties of AI technology. Currently, when we refer to AI, we usually mean techniques of machine learning (ML). ML excels at pattern recognition, especially in big data, and can process a multitude of different input types, ranging from tabular clinical data to medical imaging. These techniques have led to unprecedented advances in areas ranging from computer vision to natural language processing.But where does AI in healthcare stand today? Headlines suggest that AI has already surpassed the performance of human doctors in many medical fields [3,4]. There is no panel on AI in healthcare without the question of when AI will replace human doctors arising. These perspectives stand in stark contrast to clinical reality. Only a few AI tools are available in the clinical setting [5,6] and we are not aware of a single AI in healthcare tool appearing in clinical guidelines as a standard of care.How can this discrepancy be explained? The results gaining media attention are mostly so-called exploratory studies trying to establish proof-of-concepts (PoC) that do not reflect the full process necessary to bring these advancements to clinical reality. Exploratory studies are necessary to identify potentially valuable use cases for AI in healthcare [7]. Here, many scenarios need to be tested and successful PoCs can justify -both on an economic as well as an ethical level -the allocation of funds for follow-up confirmatory studies [7]. These confirmatory studies are needed to safely translate the exploratory results to the clinic and need to adhere to all quality markers of studies establishing efficacy, e.g. adequate power and predefined protocols [7]. Given the costs of confirmatory studies, they will only be performed as part of product development.

show abstract

Fostering reproducibility, reusability, and technology transfer in health informatics

Cited by 7 publications

References 33 publications

Guideline for software life cycle in health informatics

Guideline for software life cycle in health informatics

Guidelines and Standard Frameworks for Artificial Intelligence in Medicine: A Systematic Review

Artificial Intelligence in Healthcare: Lost In Translation?

Contact Info

Product

Resources

About