Are Regulations Safe? Reflections From Developing a Digital Cancer Decision-Support Tool

Abstract

PURPOSE Informatics solutions to early diagnosis of cancer in primary care are increasingly prevalent, but it is not clear whether existing and planned standards and regulations sufficiently address patients' safety nor whether these standards are fit for purpose. We use a patient safety perspective to reflect on the development of a computerized cancer risk assessment tool embedded within a UK primary care electronic health record system. METHODS We developed a computerized version of the CAncer Prevention in ExetER studies risk assessment tool, in compliance with the European Union's Medical Device Regulations. The process of building this tool afforded an opportunity to reflect on clinical concerns and whether current regulations for medical devices are fit for purpose. We identified concerns for patient safety and developed nine practical recommendations to mitigate these concerns. RESULTS We noted that medical device regulations (1) were initially created for hardware devices rather than software, (2) offer one-shot approval rather than supporting iterative innovation and learning, (3) are biased toward loss-transfer approaches that attempt to manage the fallout of harm instead of mitigating hazards becoming harmful, and (4) are biased toward known hazards, despite unknown hazards being an expected consequence of health care as a complex adaptive system. Our nine recommendations focus on embedding less-reductionist and stronger system perspectives into regulations and standards. CONCLUSION Our intention is to share our experience to support research-led collaborative development of health informatics solutions in cancer. We argue that regulations in the European Union do not sufficiently address the complexity of healthcare information systems with consequences for patient safety. Future standards and regulations should continue to follow a system-based approach to risk, safety, and accident avoidance.

Data-Driven Personalized Care in Lung Cancer: Scoping Review and Clinical Recommendations on Performance Status and Activity Level of Patients With Lung Cancer Using Wearable Devices

“…In particular, there are emerging concerns about safety issues of software and hardware technologies that might be used to support clinical decision-making such as cancer risk assessment. 77 This is of special importance in the context of artificial intelligence solutions where the quality of the data is a crucial element for the output of such systems in oncology, 78 , 79 and it is also linked to regulatory and governance issues. 80 In these potential future applications on the basis of extensive data collection and artificial intelligence–based outputs, we should always be careful about the acceptability by end users (eg, clinicians and patients) to avoid potential safety concerns (eg, not understanding the reliability of artificial intelligence–based algorithms) or end-user fatigue (eg, complex and confusing interfaces that generate user fatigue).…”

Machine Learning for Risk Prediction of Oesophago-Gastric Cancer in Primary Care: Comparison with Existing Risk-Assessment Tools

“…Naturally, any clinical benefits of such models would be highly dependent on model–clinician interaction, which forthcoming implementation trials such as the ERICA trial [ 22 ] may shed some light on. It is important to note that implementation phases should emphasise patient safety, adopting a systems approach to risk, safety, and accident avoidance, particularly since current regulations of health informatics solutions are likely insufficient [ 56 ].…”