Usefulness of Implementation Outcome Scales for Digital Mental Health (iOSDMH): Experiences from Six Randomized Controlled Trials

Abstract

Objectives: Measuring implementation outcomes for digital mental health interventions is essential for examining the effective delivery of these interventions. The “Implementation Outcome Scale of Digital Mental Health” (iOSDMH) has been validated and used in several trials. This study aimed to compare the iOSDMH for participants in six randomized controlled trials (RCTs) involving web-based interventions and to discuss the implications of the iOSDMH for improving the interventions. Additionally, this study examined the associations between iOSDMH scores and program completion rate (adherence). Methods: Variations in total scores and subscales of the iOSDMH were compared in six RCTs of digital mental health interventions conducted in Japan. The web-based intervention programs were based on cognitive behavioral therapy (2 programs), behavioral activation (1 program), acceptance and commitment (1 program), a combination of mindfulness, behavioral activation, and physical activity (1 program), and government guidelines for suicide prevention (1 program). Participants were full-time employees (2 programs), perinatal women (2 programs), working mothers with children (1 program), and students (1 program). The total score and subscale scores were tested using analysis of variance for between-group differences. Results: Total score and subscale scores of the iOSDMH among six trials showed a significant group difference, reflecting users’ perceptions of how each program was implemented, including aspects such as acceptability, appropriateness, feasibility, overall satisfaction, and harm. Subscale scores showed positive associations with completion rate, especially in terms of acceptability and satisfaction (R-squared = 0.93 and 0.89, respectively). Conclusions: The iOSDMH may be a useful tool for evaluating participants’ perceptions of features implemented in web-based interventions, which could contribute to improvements and further development of the intervention.

Effects of an Internet Delivered Behavioral Activation Program on Improving Work Engagement Among Japanese Workers

“…Regarding secondary outcomes, the present study found significant favorable effects on psychological distress and role-oriented future prospects, one of the subscales of well-being at work. These findings are in accordance with previous studies 13,34,36–38 . It is well-known that feeling pleasure or mastery in specific activities reduces psychological distress (ie, depression and anxiety) 16,17 and gaining mastery and self-efficacy enforces subjective well-being, such as setting and achieving goals 19–21 .…”

“…Regarding implementation outcomes, the proportion of positive responses (ie, answered “agree” or “relatively agree”) on each item were more than 60.0% on every subscale: acceptability, appropriateness, feasibility, and overall satisfaction. Compared with five programs in previous studies, 36 the length of this iBA program was highly implementable, and the rate of overall satisfaction was relatively high. Participants’ comments revealed that they might have gained self-efficacy through making action plans or reflecting on their own.…”

“…These findings are in accordance with previous studies. 13,34,[36][37][38] It is well-known that feeling pleasure or mastery in specific activities reduces psychological distress (ie, depression and anxiety) 16,17 and gaining mastery and self-efficacy enforces subjective well-being, such as setting and achieving goals. [19][20][21] This iBA program may be effective in reducing psychological distress and improving a specific aspect of eudemonic well-being at work.…”

mHealth Intervention to Promote Physical Activity Among Employees Using a Deep Learning Model for Passive Monitoring of Depression and Anxiety: Single-Arm Feasibility Trial

“…The iOSDMH for users has a total of 19 items, consisting of acceptability (3 items), appropriateness (4 items), feasibility (6 items), satisfaction (1 item), and harm (5 items). Internal consistency, structural validity, and known-class validity were confirmed in the scale development study and a previous systematic review [ 38 , 39 ]. We calculated the total scores of each subscale, and the overall score comprised acceptability, appropriateness, feasibility, and satisfaction.…”

“…As potential modifiers for the effect, the operating system in personal smartphones (iOS [reference] and Android), age (20-30 years, 40-50 years [reference], and 60 years or older), gender (male [reference], female), and the total K6 score at baseline were entered into the model as covariates. Mean scores of implementation outcomes based on iOSDMH for users were compared with those reported in previous mHealth studies [ 39 ]. All analyses were conducted at the individual level using SPSS Statistics (version 25.0, IBM Corp).…”