Evaluation Methods for Assessing Users’ Psychological Experiences of Web-Based Psychosocial Interventions: A Systematic Review

Abstract

BackgroundThe use of Web-based interventions to deliver mental health and behavior change programs is increasingly popular. They are cost-effective, accessible, and generally effective. Often these interventions concern psychologically sensitive and challenging issues, such as depression or anxiety. The process by which a person receives and experiences therapy is important to understanding therapeutic process and outcomes. While the experience of the patient or client in traditional face-to-face therapy has been evaluated in a number of ways, there appeared to be a gap in the evaluation of patient experiences of therapeutic interventions delivered online. Evaluation of Web-based artifacts has focused either on evaluation of experience from a computer Web-design perspective through usability testing or on evaluation of treatment effectiveness. Neither of these methods focuses on the psychological experience of the person while engaged in the therapeutic process.ObjectiveThis study aimed to investigate what methods, if any, have been used to evaluate the in situ psychological experience of users of Web-based self-help psychosocial interventions.MethodsA systematic literature review was undertaken of interdisciplinary databases with a focus on health and computer sciences. Studies that met a predetermined search protocol were included.ResultsAmong 21 studies identified that examined psychological experience of the user, only 1 study collected user experience in situ. The most common method of understanding users’ experience was through semistructured interviews conducted posttreatment or questionnaires administrated at the end of an intervention session. The questionnaires were usually based on standardized tools used to assess user experience with traditional face-to-face treatment.ConclusionsThere is a lack of methods specified in the literature to evaluate the interface between Web-based mental health or behavior change artifacts and users. Main limitations in the research were the nascency of the topic and cross-disciplinary nature of the field. There is a need to develop and deliver methods of understanding users’ psychological experiences while using an intervention.

Combining e‐mental health intervention development with human computer interaction (HCI) design to enhance technology‐facilitated recovery for people with depression and/or anxiety conditions: An integrative literature review

“…Online digital interventions and mobile applications for support, collaborative care, and treatment are utilized increasingly and successfully for mental health advantages (Andersson et al 2014;Feather et al 2016;Schueller et al 2017). A wide range of information is conveniently available to the general public and can be accessed using a variety of digital products and platforms (Barlow et al 2005;Huguet et al 2016;Zhao et al 2017).…”

Significant events in an Internet‐delivered (Space from Depression) intervention for depression

“…To date, however, little is known of the psychological experience of users of Internet‐delivered interventions. The psychological experience of users may be important in determining the factors influencing adherence and usage of any intervention and may reveal ways in which any intervention can be improved (Feather et al., ).…”

Participant Recruitment and Engagement in Automated eHealth Trial Registration: Challenges and Opportunities for Recruiting Women Who Experience Violence

“…Recruitment is generally advised by experts in the field to be done personally, in face-to-face, one-to-one encounters by trained, sensitive researchers who are skilled in managing unanticipated situations, and are ideally of the same culture as the women [33]. How this face-to-face recruitment process can be translated to the computer user interface in eHealth interventions has received little attention [35]. …”

“…In a review of 16 studies reporting dropout in Web-based interventions for psychological disorders, preintervention dropout (reported in 7 of the 16 studies) ranged from 4% to 52% [41]. Additionally, there is minimal theory to explain how potential participants engage with Web-based recruitment and interventions [35]. Liese and Beck [42] identified a pathway of individual and contextual factors that activate negative beliefs about the success of an intervention that are then hypothesized to lead to dropout.…”