Ethics and Law in Research on Algorithmic and Data-Driven Technology in Mental Health Care: Scoping Review

Abstract

Background Uncertainty surrounds the ethical and legal implications of algorithmic and data-driven technologies in the mental health context, including technologies characterized as artificial intelligence, machine learning, deep learning, and other forms of automation. Objective This study aims to survey empirical scholarly literature on the application of algorithmic and data-driven technologies in mental health initiatives to identify the legal and ethical issues that have been raised. Methods We searched for peer-reviewed empirical studies on the application of algorithmic technologies in mental health care in the Scopus, Embase, and Association for Computing Machinery databases. A total of 1078 relevant peer-reviewed applied studies were identified, which were narrowed to 132 empirical research papers for review based on selection criteria. Conventional content analysis was undertaken to address our aims, and this was supplemented by a keyword-in-context analysis. Results We grouped the findings into the following five categories of technology: social media (53/132, 40.1%), smartphones (37/132, 28%), sensing technology (20/132, 15.1%), chatbots (5/132, 3.8%), and miscellaneous (17/132, 12.9%). Most initiatives were directed toward detection and diagnosis. Most papers discussed privacy, mainly in terms of respecting the privacy of research participants. There was relatively little discussion of privacy in this context. A small number of studies discussed ethics directly (10/132, 7.6%) and indirectly (10/132, 7.6%). Legal issues were not substantively discussed in any studies, although some legal issues were discussed in passing (7/132, 5.3%), such as the rights of user subjects and privacy law compliance. Conclusions Ethical and legal issues tend to not be explicitly addressed in empirical studies on algorithmic and data-driven technologies in mental health initiatives. Scholars may have considered ethical or legal matters at the ethics committee or institutional review board stage. If so, this consideration seldom appears in published materials in applied research in any detail. The form itself of peer-reviewed papers that detail applied research in this field may well preclude a substantial focus on ethics and law. Regardless, we identified several concerns, including the near-complete lack of involvement of mental health service users, the scant consideration of algorithmic accountability, and the potential for overmedicalization and techno-solutionism. Most papers were published in the computer science field at the pilot or exploratory stages. Thus, these technologies could be appropriated into practice in rarely acknowledged ways, with serious legal and ethical implications.

Implementation Frameworks for Artificial Intelligence Translation Into Health Care Practice: Scoping Review

“…For example, the newly identified subdomain of shared decision-making recognizes the need for processes and behaviors that support communication, discussion, and decision-making among staff, patients, and careers (technology adopters), which is likely to be relevant for many data-driven technologies [ 37 - 39 ]. However, in the case of AI, the AI provides a fourth voice in the decision-making process that will have particular implications for how such communication is handled in an emotionally sensitive manner, how much weight is given to different opinions and preferences [ 38 , 40 ], and how it could support clinical decision-making without compromising the primary responsibilities and duties of the health care professional for patient care [ 41 ]. Similarly, the need for the evaluation of effectiveness is important for all health technologies [ 42 , 43 ], but for AI, this may be of particular importance in demonstrating the trustworthiness of data outputs if it is to replace or complement clinical judgment.…”

“…The possibility of recommending decisions across an entire population entails risks to reinforce systemic biases (eg, White or male), which might unintentionally discriminate minorities and patients with more complex or unusual health conditions. This, in turn, is linked to the increased importance of regulatory and legal systems that oversee the introduction of AI and carefully consider the implications and responsibilities for individuals, professional groups, and governments to ensure the safety, effectiveness, ethics, and equity of new data-driven technologies [ 40 ].…”

Challenges to implementing artificial intelligence in healthcare: a qualitative interview study with healthcare leaders in Sweden

“…Here, concerted and unified national authority initiatives are required according to the leaders. Despite the fact that the introduction of AI systems in healthcare appears to be inevitable, the consideration of existing regulatory and ethical mechanisms appears to be slow [ 16 , 18 ]. Additionally, another challenge attributable to the setting was the lack of to increase the competence and expertise among professionals in AI systems, which could be a potential barrier to the implementation of AI in practice.…”

“…There is, however, a current research gap between the development of robust algorithms and the implementation of AI systems in healthcare practice. The conclusion in newly published reviews addressing regulation, privacy and legal aspects [ 15 , 16 ], ethics [ 16 – 18 ], clinical and patient outcomes [ 19 – 21 ] and economic impact [ 22 ], is that further research is needed in a real-world clinical setting although the clinical implementation of AI technology is still at an early stage. There are no studies describing implementation frameworks or models that could inform us concerning the role of barriers and facilitators in the implementation process and relevant implementation strategies of AI technology [ 23 ].…”

Impact of Electronic Health Records on Information Practices in Mental Health Contexts: Scoping Review

“…The potential impact of this secondary data use makes it increasingly urgent to address the issues raised in this study. Secondary data use in the mental health context requires further ethical consideration, especially as new data sources are being introduced into the health care system, such as data from wearables [ 151 , 152 ].…”