Technology-Enabled Solutions for Australian Mental Health Services Reform: Impact Evaluation (Preprint)

Abstract

BACKGROUND Health information technologies (HITs) are becoming increasingly recognised for their potential to provide innovative solutions to improve the delivery of mental health services and drive system reform for better outcomes. A growing body of research aims to evaluate how innovative HITs can be co-designed and successfully implemented in mental health services to enhance their provision of care. OBJECTIVE The objective of this paper is to describe baseline results for a study designed to systematically monitor and evaluate the impact of implementing HIT-enabled solutions into Australian mental health services, with the aim of facilitating the iterative refinement of these solutions to meet the needs of consumers and their supportive others as well as health professionals and service providers. METHODS Data were collected via Web-based surveys, semi-structured interviews, and a workshop with staff from the mental health services implementing the HIT-enabled solutions to systematically monitor and evaluate their impact. Descriptive statistics and Fisher exact tests were used to characterise the findings from the Web-based surveys, including variability in the results based on service. Semi-structured interviews were coded using thematic analysis and workshop data was coded using basic content analysis. RESULTS At the end of the first phase of impact evaluation, baseline data had been collected from staff from three primary youth mental health services (n=18), a counselling service for veterans and their families (n=23), and a helpline for individuals affected by eating disorders and negative body image issues (n=6). As reported via Web-based surveys, staff members across the services consistently “agreed” or “strongly agreed” there was benefit associated with using technology as part of their work (80.9%) and that HIT-enabled solutions have the potential to improve outcomes for consumers (60.0%); however, there was less certainty as to whether their consumers’ capability to use technology aligned with how solutions would be used as part of their mental health care (24.4% “strongly disagreed” or “disagreed”; 33.3% “neutral”; and 42.2% “strongly agreed” or “agreed”). Implementation barriers and facilitators for consumers, health professionals and at the service level were derived from the baseline semi-structured interviews (n=3) and a workshop. Participants consistently indicated that the HIT-enabled solution was appropriate for their respective services; however, questioned whether their consumers had the digital literacy required to use the technology. Additional potential barriers to implementation included health professional’s digital literacy as well as service readiness for change in relation to adopting the HIT-enabled model of care. CONCLUSIONS Baseline data from the first phase of data collection highlight consistent themes related to the successful implementation of HIT-enabled solutions in mental health services, including the value of assessing service readiness for change (e.g. existing technology infrastructure and the digital literacy of staff and consumers), the importance of leadership from senior service management and local champions to drive quality improvement, and the necessity for effective education and training in the context of continuous on the ground support. Longitudinal data will provide a deeper understanding of the impact of HIT-enabled solutions on service quality and digital readiness and competence as well as the social return on investment and the quality, usability and acceptability of the solution.

Informing the Future of Integrated Digital and Clinical Mental Health Care: Synthesis of the Outcomes From Project Synergy

“…As an example, Titov et al [9] have reported success in integrating internet-delivered cognitive behavioral therapy as part of clinical care with a mental health professional in 5 clinics internationally. Unfortunately, far more studies highlight barriers to successful implementations of digital health solutions [10][11][12][13], and further research is required to identify the most effective ways in which to design and implement evidence-based HITs as part of clinical care at scale.…”

“…To that end, Mohr et al [29] highlight the benefits of evaluating the challenges and successes associated with tools such as the InnoWell Platform in targeted clinical settings, such as through the Project Synergy participating services, to iteratively refine and optimize usability, acceptability, and effectiveness to ensure that it meets identified clinical objectives. Therefore, as detailed previously by LaMonica et al [10], we used a mixed methods impact evaluation study, including web-based surveys, semistructured interviews, and workshops, to identify potential barriers to and facilitators of implementation and to evaluate the impact of the InnoWell Platform on the participating mental health services over time, including (1) digital literacy and competence of the service staff in relation to the implementation of the HIT in the service;…”

“…Staff across multiple service settings consistently support the use of technology as part of their work; however, they also list digital literacy of both service users and health professionals as well as service readiness for change as potential barriers to widespread adoption [10].…”

“…To facilitate a successful implementation, service leadership need to establish clear strategies to mitigate potential barriers to implementation identified by on-the-ground staff responsible for driving the implementation [10].…”

“…Health professionals are often confident about the effectiveness of their current service models (ie, business as usual) and express reluctance to change their usual practices [10].…”