Assessing the usability and user engagement of Thought Spot - A digital mental health help-seeking solution for transition-aged youth

Abstract

Objective To evaluate the perceived usability of and user engagement with a digital platform (Thought Spot) designed to enhance mental health and wellness help-seeking among transition-aged youth (TAY; 17–29-years old). Materials and methods Survey responses and usage patterns were collected as part of a randomized controlled trial evaluating the efficacy of Thought Spot. Participants given Thought Spot completed an adapted Usefulness, Satisfaction, and Ease of Use (USE) Questionnaire to measure perceived usability of the platform. User engagement patterns on Thought Spot were examined using analytics data collected throughout the study (March 2018–June 2019). Results A total of 131 transition-aged participants completed the USE questionnaire and logged on to Thought Spot at least once. Ease of learning scored higher than ease of use, usefulness and satisfaction. Participants identified numerous strengths and challenges related to usability, visual appeal, functionality and usefulness of the content. In terms of user engagement, most participants stopped using the platform after 3 weeks. Participants searched and were interested in a variety of resources, including mental health, counselling and social services. Discussion Participants reported mixed experiences while using Thought Spot and exhibited low levels of long-term user engagement. User satisfaction, the willingness to recommend Thought Spot to others, and the willingness for future use appeared to be influenced by content relevance, ease of learning, available features, and other contextual factors. Analysis of the types of resources viewed and searches conducted by TAY end-users provided insight into their behaviour and needs. Conclusion Users had mixed perceptions about the usability of Thought Spot, which may have contributed to the high attrition rate. User satisfaction and engagement appears to be influenced by content relevance, ease of learning, and the types of features available. Further investigation to understand the contextual factors that affect TAYs' adoption and engagement with digital mental health tools is required.

Online mental health interventions designed for students in higher education: A user-centered perspective

“… Theme Theme Found in Participants’ Comment Example Convenience ( Doherty et al, 2012 ; Farrer et al, 2020 ; Morr et al, 2020 ) “I would imagine that for me an online community takes a lot of excuses I could have, like I don't want to go, or I can't fit it into my schedule, I can always go online” ( Morr et al, 2020 ) Add personalization ( Huang et al, 2015 ; Hickie et al, 2019 ; Meng et al, 2018 ; Farrer et al, 2020 ; Morr et al, 2020 ) “My roommate and I have different style of meditation. Like, for me, I prefer like a guided meditation that talks about sensation, whereas she prefers more like, a visualization, like imagine yourself on a beach, kind of thing, so having those options will be good.” ( Morr et al, 2020 ) Improve interface/presentation ( Báldy et al, 2020 ; Currie et al, 2010 ; Farrer et al, 5 2020 ; Morr et al, 2020 ; Shi et al, 2021 ) “Improve the animation quality. Change the camera shaking” ( Báldy et al, 2020 ) Change format of intervention ( Báldy et al, 2020 ; Papadatou-Pastou et al, 2019 ) “Make it a Virtual Reality game” ( Báldy et al, 2020 ) Change intervention ( Huang et al, 2015 ; Ananthabhotla et al, 2017 ; Doherty et al, 2012 ) “I don't like to think back.…”

“…I only want to look at the present. I don't see the report has value to me” ( Huang et al, 2015 ) Improve content ( Kim et al, 2020 ; Huang et al, 2015 ; Ananthabhotla et al, 2017 ; Hickie et al, 2019 ; Meng et al, 2018 ; Farrer et al, 2020 ; Shi et al, 2021 ) “I tend to eat more when I'm more depressed, more of a comfort food. Visualization will be helpful so that I can see really highs or lows” ( Meng et al, 2018 ) Ensure anonymity/privacy/safety ( Doherty et al, 2012 ; Meng et al, 2018 ; Gulliver et al, 2015 ; Farrer et al, 2020 ; Morr et al, 2020 ) “I think it's important to have your username account but also a guest account so if you don't want to do something which someone else can see, you can use the private guest account, which won't record your information.” ( Gulliver et al, 2015 ) Add peer engagement ( Huang et al, 2015 ; Doherty et al, 2012 ; Farrer et al, 2020 ; Morr et al, 2020 ) “I'd like to have more people using the app.…”

“…For ThoughtSpot ( Wiljer et al, 2020 ), the authors report using a usability questionnaire with open-ended questions in their study, however, the results of that survey were recently published in a separate research article on March 26, 2021. We include the details of that study in the current analysis ( Shi et al, 2021 ).…”

“…Some example comments from students include “Make it engaging and fun”( Coughlan et al, 2019 ), “Add regular content updates” ( Farrer et al, 2020 ) etc. Similarly, the theme “Improve Content” shows that students interact sufficiently enough to suggest improvements to the content of the interventions ( Kim et al, 2020 ; Huang et al, 2015 ; Ananthabhotla et al, 2017 ; Hickie et al, 2019 ; Meng et al, 2018 ; Farrer et al, 2020 ; Shi et al, 2021 ).…”

Digital Mental Health Amid COVID-19

“…An automated randomized controlled trial of an online peer-support program (Big White Wall) versus web-based information for self-management of anxiety and depression suggested a personal approach for participant engagement to explain the benefits of this novel intervention [33]. It was proposed that user engagement and satisfaction with a digital platform (Thought Spot) be related to content relevance, ease of learning, and available features [34]. The effectiveness of a digital platform with virtual coaches for cognitive behavior therapy (CBT) was explored in a qualitative study-it was important to find ways for users to be drawn to and then consistently engaged with the digital platform [35].…”