Stealth research: Lack of peer‐reviewed evidence from healthcare unicorns

Cristea, Ioana A.; Cahan, Eli M.; Ioannidis, John P. A.

doi:10.1111/eci.13072

Cited by 37 publications

(17 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Unfortunately, many products in Table 2 are pursuing 'stealth science' to protect trade secrets and avoiding regulatory or academic scrutiny. 99 While stealth science is not uncommon amongst biomedical innovations, lack of transparency is particularly concerning with machine learning. This narrative review was unable to provide standard metrics of adoption, because many of the figures marketed by product developers have no peer-reviewed evidence.…”

Section: Challenges and Opportunitiesmentioning

confidence: 99%

A Path for Translation of Machine Learning Products into Healthcare Delivery

Sendak¹,

D’Arcy²,

Kashyap³

et al. 2020

EMJ Innov

View full text Add to dashboard Cite

Despite enormous enthusiasm, machine learning models are rarely translated into clinical care and there is minimal evidence of clinical or economic impact. New conference venues and academic journals have emerged to promote the proliferating research; however, the translational path remains unclear. This review undertakes the first in-depth study to identify how machine learning models that ingest structured electronic health record data can be applied to clinical decision support tasks and translated into clinical practice. The authors complement their own work with the experience of 21 machine learning products that address problems across clinical domains and across geographic populations. Four phases of translation emerge: design and develop, evaluate and validate, diffuse and scale, and continuing monitoring and maintenance. The review highlights the varying approaches taken across each phase by teams building machine learning products and presents a discussion of challenges and opportunities. The translational path and associated findings are instructive to researchers and developers building machine learning products, policy makers regulating machine learning products, and health system leaders who are considering adopting a machine learning product.

show abstract

Section: Challenges and Opportunitiesmentioning

confidence: 99%

A Path for Translation of Machine Learning Products into Healthcare Delivery

Sendak¹,

D’Arcy²,

Kashyap³

et al. 2020

EMJ Innov

View full text Add to dashboard Cite

show abstract

“…Placing these general considerations into perspective, Cristea and colleagues recently evaluated the lack of peer-reviewed evidence from healthcare unicorns 6 ; that is, biotech startups with promising technologies and market valuations that exceed the 1 billion threshold. Many startups had secured exceedingly high market valuations despite a lack of peer-reviewed publications to support their market-directed innovations.…”

mentioning

confidence: 99%

“…Scientific advisory boards are also important to guarantee field expertise. However, information on the membership and level of involvement of these boards is often lacking 6 .…”

mentioning

confidence: 99%

‘Stealth’ corporate innovation: an emerging threat for therapeutic drug development

et al. 2019

Self Cite

View full text Add to dashboard Cite

“…As highlighted by regulatory agencies such as the National Institute for health and Care Excellence (NICE) [30,31] and the US Food and Drug Administration (FDA) [32,33], AI-driven CDSSs will usually require a high standard of clinical evidence before they can be safely used. However, there are now a large number of digital health startups operating in "stealth mode", with large investments and research and development programmes but without published evidence of mechanism of action or clinical effectiveness [34]. This conflict is likely to be resolved as new regulations are implemented, but vigilance will be required by clinicians, researchers, and policy-makers to ensure that open research approaches are adopted so that evaluations of AI systems are clinically relevant and reproducible, and can enable informed decisions about whether or not they should be adopted.…”

Section: Open Researchmentioning

confidence: 99%

An Open Science Approach to Artificial Intelligence in Healthcare

Paton

Kobayashi

2019

Yearb Med Inform

View full text Add to dashboard Cite

Objectives: Artificial Intelligence (AI) offers significant potential for improving healthcare. This paper discusses how an “open science” approach to AI tool development, data sharing, education, and research can support the clinical adoption of AI systems. Method: In response to the call for participation for the 2019 International Medical Informatics Association (IMIA) Yearbook theme issue on AI in healthcare, the IMIA Open Source Working Group conducted a rapid review of recent literature relating to open science and AI in healthcare and discussed how an open science approach could help overcome concerns about the adoption of new AI technology in healthcare settings. Results: The recent literature reveals that open science approaches to AI system development are well established. The ecosystem of software development, data sharing, education, and research in the AI community has, in general, adopted an open science ethos that has driven much of the recent innovation and adoption of new AI techniques. However, within the healthcare domain, adoption may be inhibited by the use of “black-box” AI systems, where only the inputs and outputs of those systems are understood, and clinical effectiveness and implementation studies are missing. Conclusions: As AI-based data analysis and clinical decision support systems begin to be implemented in healthcare systems around the world, further openness of clinical effectiveness and mechanisms of action may be required by safety-conscious healthcare policy-makers to ensure they are clinically effective in real world use.

show abstract

Stealth research: Lack of peer‐reviewed evidence from healthcare unicorns

Cited by 37 publications

References 7 publications

A Path for Translation of Machine Learning Products into Healthcare Delivery

A Path for Translation of Machine Learning Products into Healthcare Delivery

‘Stealth’ corporate innovation: an emerging threat for therapeutic drug development

An Open Science Approach to Artificial Intelligence in Healthcare

Contact Info

Product

Resources

About